Le but de ce notebook est d'effectuer de la classification d'images de chiens sur leur race. Différents traitements sont effectués et plusieurs modèles sont entraînés. Le transfert learning est également présent par l'utilisation de VGG16 et de NASNET.
import os
import numpy as np
import xml.etree.ElementTree as ET
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
from PIL import Image, ImageOps
from sklearn.preprocessing import LabelEncoder
from keras.utils.np_utils import to_categorical
from sklearn.model_selection import train_test_split
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator, load_img, img_to_array
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense,\
BatchNormalization, Dropout, GlobalAveragePooling2D
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.initializers import he_uniform
from tensorboard.plugins.hparams import api as hp
import datetime
import warnings
warnings.filterwarnings('ignore')
np.random.seed(1)
%load_ext tensorboard
!rm -rf ./logs/
Récupération des données brutes
!unzip 'drive/My Drive/OC/Projet 6.zip'
Récupération des données traitées
!unzip 'drive/My Drive/OC/data.zip'
races_list = os.listdir("data")
races_list = os.listdir("Images")
annotions_list = os.listdir("Annotation")
print("{} races de chiens".format(len(races_list)))
print(races_list[:5])
120 races de chiens ['Great_Pyrenees', 'cairn', 'Old_English_sheepdog', 'Welsh_springer_spaniel', 'basset']
Le dataset comprends plus de 20000 images de chiens appartenant à 120 races différentes.
Affichons quelques images de la race "coated_wheaten_terrier"
plt.figure(figsize=(10,5))
images = os.listdir("Images/"+races_list[0])[:8]
for i in range(8):
img = mpimg.imread("Images/"+races_list[0] + '/'+ images[i])
plt.subplot(2, 4, i+1)
plt.imshow(img)
plt.axis('off')
Nous pouvons remarquer que les images montrent bien chacunes un chien. Cependant nous notons qu'il y a beaucoup d'informations superflues étant donné que les chiens ne sont pas totalement centrés.
Ainsi nous allons utiliser l'autre dataset donné que indique la localisation du chien dans chaque image.
plt.figure(figsize=(10,5))
for i, image in enumerate(images):
img = Image.open('Images/{}/{}'.format(races_list[0], image))
tree = ET.parse('Annotation/{}/{}'.format(races_list[0], image).replace(".jpg",""))
xmin = int(tree.getroot().findall('object')[0].find('bndbox').find('xmin').text)
xmax = int(tree.getroot().findall('object')[0].find('bndbox').find('xmax').text)
ymin = int(tree.getroot().findall('object')[0].find('bndbox').find('ymin').text)
ymax = int(tree.getroot().findall('object')[0].find('bndbox').find('ymax').text)
img = img.crop((xmin, ymin, xmax, ymax))
img = img.convert('RGB')
plt.subplot(2, 4, i+1)
plt.imshow(img)
plt.axis('off')
Les images sont bien centrées sur le chien permettant ainsi de mieux entraîner nos modèles.
Effectuons ces traitements sur l'ensemble des images et sauvegardons-les pour les prochaines utilisations.
os.mkdir('data')
for races in races_list:
race = races.split("-")[1]
os.mkdir('data/' + race)
for image in os.listdir("Images/{}".format(races)):
img = Image.open('Images/{}/{}'.format(races, image))
tree = ET.parse('Annotation/{}/{}'.format(races, image).replace(".jpg",""))
xmin = int(tree.getroot().findall('object')[0].find('bndbox').find('xmin').text)
xmax = int(tree.getroot().findall('object')[0].find('bndbox').find('xmax').text)
ymin = int(tree.getroot().findall('object')[0].find('bndbox').find('ymin').text)
ymax = int(tree.getroot().findall('object')[0].find('bndbox').find('ymax').text)
img = img.crop((xmin, ymin, xmax, ymax))
img = img.resize((224, 224))
img = img.convert('RGB')
img.save('data/' + race + '/' + image)
Regardons la distribution des classes
dict_race = {}
for race in os.listdir("data"):
dict_race[race] = len(os.listdir("data/"+race))
plt.bar(dict_race.keys(), dict_race.values())
<BarContainer object of 120 artists>
Chaque race a environ 150 échantillons de races. Le jeu de données est plutôt bien balancé.
Commençons à générer un dataset pour l'entraînement et l'autre de validation.
Ensuite, construisons un réseau de neurones assez simple
datagen_base = ImageDataGenerator(rescale=1/255,
validation_split=0.3)
train_data = datagen_base.flow_from_directory(batch_size=32,
directory=r'data',
shuffle=True,
seed = 1,
target_size=(224, 224),
color_mode="rgb",
subset="training",
class_mode='categorical'
)
validation_data = datagen_base.flow_from_directory(batch_size=32,
directory=r'data',
shuffle=True,
seed = 1,
target_size=(224, 224),
color_mode="rgb",
subset="validation",
class_mode='categorical')
Found 14458 images belonging to 120 classes. Found 6122 images belonging to 120 classes.
model = tf.keras.models.Sequential([
Conv2D(32, (3, 3), activation='relu', input_shape=(224, 224, 3)),
MaxPooling2D(2,2),
Conv2D(64, (3, 3), activation='relu'),
MaxPooling2D(2,2),
Conv2D(128, (3, 3), activation='relu'),
MaxPooling2D(2,2),
Conv2D(128, (3, 3), activation='relu'),
MaxPooling2D(2,2),
Flatten(),
Dense(512, activation='relu'),
Dense(len(races_list), activation='softmax')
])
model.summary()
Model: "sequential_5" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d_24 (Conv2D) (None, 222, 222, 32) 896 _________________________________________________________________ max_pooling2d_24 (MaxPooling (None, 111, 111, 32) 0 _________________________________________________________________ conv2d_25 (Conv2D) (None, 109, 109, 64) 18496 _________________________________________________________________ max_pooling2d_25 (MaxPooling (None, 54, 54, 64) 0 _________________________________________________________________ conv2d_26 (Conv2D) (None, 52, 52, 128) 73856 _________________________________________________________________ max_pooling2d_26 (MaxPooling (None, 26, 26, 128) 0 _________________________________________________________________ conv2d_27 (Conv2D) (None, 24, 24, 128) 147584 _________________________________________________________________ max_pooling2d_27 (MaxPooling (None, 12, 12, 128) 0 _________________________________________________________________ flatten_6 (Flatten) (None, 18432) 0 _________________________________________________________________ dense_11 (Dense) (None, 512) 9437696 _________________________________________________________________ dense_12 (Dense) (None, 120) 61560 ================================================================= Total params: 9,740,088 Trainable params: 9,740,088 Non-trainable params: 0 _________________________________________________________________
Ce CNN a 5 couches et possède presque 10 millions de paramètres.
model.compile(optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'])
log_dir = "logs/fit/Base_Model"
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
model.fit_generator(generator=train_data,
validation_data=validation_data,
steps_per_epoch=len(train_data),
validation_steps=len(validation_data),
epochs=10,
callbacks=[tensorboard_callback]
)
Epoch 1/10 452/452 [==============================] - 38s 84ms/step - loss: 4.6897 - accuracy: 0.0163 - val_loss: 4.4696 - val_accuracy: 0.0343 Epoch 2/10 452/452 [==============================] - 38s 84ms/step - loss: 4.1044 - accuracy: 0.0724 - val_loss: 4.0030 - val_accuracy: 0.0835 Epoch 3/10 452/452 [==============================] - 38s 83ms/step - loss: 3.6289 - accuracy: 0.1419 - val_loss: 3.8540 - val_accuracy: 0.1103 Epoch 4/10 452/452 [==============================] - 38s 84ms/step - loss: 3.0869 - accuracy: 0.2439 - val_loss: 3.9643 - val_accuracy: 0.1312 Epoch 5/10 452/452 [==============================] - 37s 83ms/step - loss: 2.3363 - accuracy: 0.4056 - val_loss: 4.2611 - val_accuracy: 0.1184 Epoch 6/10 452/452 [==============================] - 38s 84ms/step - loss: 1.4972 - accuracy: 0.6042 - val_loss: 5.3713 - val_accuracy: 0.1094 Epoch 7/10 452/452 [==============================] - 38s 84ms/step - loss: 0.8137 - accuracy: 0.7789 - val_loss: 7.1578 - val_accuracy: 0.1088 Epoch 8/10 452/452 [==============================] - 38s 84ms/step - loss: 0.4193 - accuracy: 0.8843 - val_loss: 8.9363 - val_accuracy: 0.1086 Epoch 9/10 452/452 [==============================] - 38s 83ms/step - loss: 0.2734 - accuracy: 0.9234 - val_loss: 10.3452 - val_accuracy: 0.1039 Epoch 10/10 452/452 [==============================] - 38s 83ms/step - loss: 0.2327 - accuracy: 0.9390 - val_loss: 11.6367 - val_accuracy: 0.0995
<tensorflow.python.keras.callbacks.History at 0x7f1f4cd0c550>
Au bout de 10 epochs, le CNN atteint 90% en taux de succès pour l'échantillon d'entraînement. Cependant, ce score n'est que de 13% pour l'échantillon de validation.
Nous sommes dans un cas de surapprentissage donc nous allons ajouter de la régularisation.
model = tf.keras.models.Sequential([
Conv2D(32, (3, 3), activation='relu', input_shape=(224, 224, 3)),
BatchNormalization(),
MaxPooling2D(2,2),
Conv2D(64, (3, 3), activation='relu'),
BatchNormalization(),
MaxPooling2D(2,2),
Conv2D(128, (3, 3), activation='relu'),
BatchNormalization(),
MaxPooling2D(2,2),
Conv2D(128, (3, 3), activation='relu'),
BatchNormalization(),
MaxPooling2D(2,2),
Flatten(),
Dense(512, activation='relu'),
Dropout(0.5),
Dense(len(races_list), activation='softmax')
])
model.compile(optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'])
log_dir = "logs/fit/Base_Model_Regularization"
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
model.fit_generator(generator=train_data,
validation_data=validation_data,
steps_per_epoch=len(train_data),
validation_steps=len(validation_data),
epochs=10,
callbacks=[tensorboard_callback]
)
Epoch 1/10 452/452 [==============================] - 37s 82ms/step - loss: 5.0510 - accuracy: 0.0119 - val_loss: 4.7827 - val_accuracy: 0.0127 Epoch 2/10 452/452 [==============================] - 37s 82ms/step - loss: 4.7883 - accuracy: 0.0118 - val_loss: 4.7820 - val_accuracy: 0.0108 Epoch 3/10 452/452 [==============================] - 37s 82ms/step - loss: 4.7888 - accuracy: 0.0122 - val_loss: 4.7830 - val_accuracy: 0.0131 Epoch 4/10 452/452 [==============================] - 38s 83ms/step - loss: 4.7929 - accuracy: 0.0124 - val_loss: 4.7839 - val_accuracy: 0.0106 Epoch 5/10 452/452 [==============================] - 38s 85ms/step - loss: 4.7812 - accuracy: 0.0122 - val_loss: 4.7780 - val_accuracy: 0.0131 Epoch 6/10 452/452 [==============================] - 38s 84ms/step - loss: 4.7804 - accuracy: 0.0122 - val_loss: 4.7798 - val_accuracy: 0.0123 Epoch 7/10 452/452 [==============================] - 38s 84ms/step - loss: 4.7800 - accuracy: 0.0122 - val_loss: 4.7797 - val_accuracy: 0.0121 Epoch 8/10 452/452 [==============================] - 38s 84ms/step - loss: 4.7800 - accuracy: 0.0122 - val_loss: 4.7791 - val_accuracy: 0.0119 Epoch 9/10 452/452 [==============================] - 38s 84ms/step - loss: 4.7799 - accuracy: 0.0122 - val_loss: 4.7778 - val_accuracy: 0.0132 Epoch 10/10 452/452 [==============================] - 38s 83ms/step - loss: 4.7797 - accuracy: 0.0124 - val_loss: 4.7789 - val_accuracy: 0.0113
<tensorflow.python.keras.callbacks.History at 0x7f1f4a8e4c88>
Les taux de succès des deux jeux de données sont maintenant sensiblement égaux. Cependant, ils n'atteignent qu'un peu plus de 1%.
Pour améliorer ce score, nous pouvons employer de l'augmentation de données.
Nous allons employer la data augmentation sur le jeu de données d'entraînement en effectuant plusieurs transformations tels que des zoomages, des rotations, des changements d'intensités lumineuses, etc.
Regardons un exemple à partir de cette image
img = mpimg.imread("data/Airedale/n02096051_129.jpg")
plt.imshow(img)
<matplotlib.image.AxesImage at 0x7f1f5018aba8>
datagen = ImageDataGenerator(zoom_range=0.1,
brightness_range=[0.7,1.2],
rotation_range=25,
horizontal_flip=True,
zca_whitening = True,
height_shift_range=0.05,
width_shift_range=0.05)
iter_data = datagen.flow(np.expand_dims(img, 0), batch_size=1)
plt.figure(figsize=(12,8))
for i in range(12):
plt.subplot(3, 4, i+1)
plt.imshow(iter_data.next()[0].astype('int'))
plt.axis('off')
L'image initiale a été transformée aléatoirement 12 fois ci-dessus. Nous observons bien les différences de luminosité ou les rotations effectuées.
datagen = ImageDataGenerator(rescale=1/255,
validation_split=0.3,
zoom_range=0.1,
brightness_range=[0.7,1.2],
rotation_range=25,
horizontal_flip=True,
zca_whitening = True,
height_shift_range=0.05,
width_shift_range=0.05)
datagen_val = ImageDataGenerator(rescale=1/255,
validation_split=0.3)
train_data = datagen.flow_from_directory(batch_size=32,
directory=r'data',
shuffle=True,
seed = 1,
target_size=(224, 224),
color_mode="rgb",
subset="training",
class_mode='categorical'
)
validation_data = datagen_val.flow_from_directory(batch_size=32,
directory=r'data',
shuffle=True,
seed = 1,
target_size=(224, 224),
color_mode="rgb",
subset="validation",
class_mode='categorical')
Found 14458 images belonging to 120 classes. Found 6122 images belonging to 120 classes.
model = tf.keras.models.Sequential([
Conv2D(32, (3, 3), activation='relu', input_shape=(224, 224, 3)),
MaxPooling2D(2,2),
Conv2D(64, (3, 3), activation='relu'),
MaxPooling2D(2,2),
Conv2D(128, (3, 3), activation='relu'),
MaxPooling2D(2,2),
Conv2D(128, (3, 3), activation='relu'),
MaxPooling2D(2,2),
Flatten(),
Dense(512, activation='relu'),
Dense(len(races_list), activation='softmax')
])
model.compile(optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'])
model.summary()
Model: "sequential_7" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d_32 (Conv2D) (None, 222, 222, 32) 896 _________________________________________________________________ max_pooling2d_32 (MaxPooling (None, 111, 111, 32) 0 _________________________________________________________________ conv2d_33 (Conv2D) (None, 109, 109, 64) 18496 _________________________________________________________________ max_pooling2d_33 (MaxPooling (None, 54, 54, 64) 0 _________________________________________________________________ conv2d_34 (Conv2D) (None, 52, 52, 128) 73856 _________________________________________________________________ max_pooling2d_34 (MaxPooling (None, 26, 26, 128) 0 _________________________________________________________________ conv2d_35 (Conv2D) (None, 24, 24, 128) 147584 _________________________________________________________________ max_pooling2d_35 (MaxPooling (None, 12, 12, 128) 0 _________________________________________________________________ flatten_8 (Flatten) (None, 18432) 0 _________________________________________________________________ dense_15 (Dense) (None, 512) 9437696 _________________________________________________________________ dense_16 (Dense) (None, 120) 61560 ================================================================= Total params: 9,740,088 Trainable params: 9,740,088 Non-trainable params: 0 _________________________________________________________________
En effectuant des tests au préalable, mettre le learning_rate à 0.001 semble être favorable au modèle.
De plus, comme nous utilisons la fonction d'activation ReLu, il est préférable d'initialiser les paramètres avec l'initialisation de He.
model = tf.keras.models.Sequential([
Conv2D(32, (3, 3), activation='relu', input_shape=(224, 224, 3), kernel_initializer=he_uniform()),
MaxPooling2D(2,2),
Conv2D(64, (3, 3), activation='relu', kernel_initializer=he_uniform()),
MaxPooling2D(2,2),
Conv2D(128, (3, 3), activation='relu', kernel_initializer=he_uniform()),
MaxPooling2D(2,2),
Conv2D(128, (3, 3), activation='relu', kernel_initializer=he_uniform()),
MaxPooling2D(2,2),
Flatten(),
Dense(512, activation='relu', kernel_initializer=he_uniform()),
Dense(len(races_list), activation='softmax')
])
model.compile(optimizer=Adam(learning_rate=0.001),
loss='categorical_crossentropy',
metrics=['accuracy'])
log_dir = "logs/fit/Base_Model_DataAugmentation"
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
model.fit_generator(generator=train_data,
validation_data=validation_data,
steps_per_epoch=len(train_data),
validation_steps=len(validation_data),
epochs=50,
callbacks=[tensorboard_callback]
)
Au bout d'une vingtaine d'epoch, la fonction d'objectif se détériore ainsi nous pouvons arrêtre l'exécition à 25 epoch.
Etant donné qu'aucune technique de régularisation n'est employée, le taux de succès du dataset d'entraînement est meilleur que celui de validation.
Cependant, avec la data augmentation, nous parvenons à obtenir un taux de succès d'environ 26%.
Entraînons maintenant des modèles plus profonds.
Nous allons nous inspirer du modèle VGG16 avec cependant moins de couches.
model = tf.keras.models.Sequential([
Conv2D(32, (3, 3), activation='relu', input_shape=(224, 224, 3), kernel_initializer=he_uniform()),
Conv2D(64, (3, 3), activation='relu', kernel_initializer=he_uniform()),
MaxPooling2D(2,2),
BatchNormalization(),
Conv2D(64, (3, 3), activation='relu', kernel_initializer=he_uniform()),
Conv2D(128, (3, 3), activation='relu', kernel_initializer=he_uniform()),
MaxPooling2D(2,2),
BatchNormalization(),
Conv2D(128, (3, 3), activation='relu', kernel_initializer=he_uniform()),
Conv2D(256, (3, 3), activation='relu', kernel_initializer=he_uniform()),
MaxPooling2D(2,2),
BatchNormalization(),
Conv2D(256, (3, 3), activation='relu', kernel_initializer=he_uniform()),
Conv2D(512, (3, 3), activation='relu', kernel_initializer=he_uniform()),
Conv2D(512, (3, 3), activation='relu', kernel_initializer=he_uniform()),
MaxPooling2D(2,2),
BatchNormalization(),
Conv2D(512, (3, 3), activation='relu', kernel_initializer=he_uniform()),
Conv2D(512, (3, 3), activation='relu', kernel_initializer=he_uniform()),
Conv2D(256, (3, 3), activation='relu', kernel_initializer=he_uniform()),
MaxPooling2D(2,2),
BatchNormalization(),
Flatten(),
Dense(512, activation='relu', kernel_initializer=he_uniform()),
Dropout(0.5),
Dense(256, activation='relu', kernel_initializer=he_uniform()),
Dropout(0.5),
Dense(len(races_list), activation='softmax')
])
print(model.summary())
Model: "sequential_11" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d_64 (Conv2D) (None, 222, 222, 32) 896 _________________________________________________________________ conv2d_65 (Conv2D) (None, 220, 220, 64) 18496 _________________________________________________________________ max_pooling2d_50 (MaxPooling (None, 110, 110, 64) 0 _________________________________________________________________ batch_normalization_30 (Batc (None, 110, 110, 64) 256 _________________________________________________________________ conv2d_66 (Conv2D) (None, 108, 108, 64) 36928 _________________________________________________________________ conv2d_67 (Conv2D) (None, 106, 106, 128) 73856 _________________________________________________________________ max_pooling2d_51 (MaxPooling (None, 53, 53, 128) 0 _________________________________________________________________ batch_normalization_31 (Batc (None, 53, 53, 128) 512 _________________________________________________________________ conv2d_68 (Conv2D) (None, 51, 51, 128) 147584 _________________________________________________________________ conv2d_69 (Conv2D) (None, 49, 49, 256) 295168 _________________________________________________________________ max_pooling2d_52 (MaxPooling (None, 24, 24, 256) 0 _________________________________________________________________ batch_normalization_32 (Batc (None, 24, 24, 256) 1024 _________________________________________________________________ conv2d_70 (Conv2D) (None, 22, 22, 256) 590080 _________________________________________________________________ conv2d_71 (Conv2D) (None, 20, 20, 512) 1180160 _________________________________________________________________ conv2d_72 (Conv2D) (None, 18, 18, 512) 2359808 _________________________________________________________________ max_pooling2d_53 (MaxPooling (None, 9, 9, 512) 0 _________________________________________________________________ batch_normalization_33 (Batc (None, 9, 9, 512) 2048 _________________________________________________________________ conv2d_73 (Conv2D) (None, 7, 7, 512) 2359808 _________________________________________________________________ conv2d_74 (Conv2D) (None, 5, 5, 512) 2359808 _________________________________________________________________ conv2d_75 (Conv2D) (None, 3, 3, 256) 1179904 _________________________________________________________________ max_pooling2d_54 (MaxPooling (None, 1, 1, 256) 0 _________________________________________________________________ batch_normalization_34 (Batc (None, 1, 1, 256) 1024 _________________________________________________________________ flatten_12 (Flatten) (None, 256) 0 _________________________________________________________________ dense_25 (Dense) (None, 512) 131584 _________________________________________________________________ dropout_7 (Dropout) (None, 512) 0 _________________________________________________________________ dense_26 (Dense) (None, 256) 131328 _________________________________________________________________ dropout_8 (Dropout) (None, 256) 0 _________________________________________________________________ dense_27 (Dense) (None, 120) 30840 ================================================================= Total params: 10,901,112 Trainable params: 10,898,680 Non-trainable params: 2,432 _________________________________________________________________ None
Ce modèle a 14 couches, soit presque 3 fois plus que le CNN précédent, avec uniquement 1 million de paramètre en plus.
model.compile(optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'])
log_dir = "logs/fit/Deep_Model"
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
model.fit_generator(generator=train_data,
validation_data=validation_data,
steps_per_epoch=len(train_data),
validation_steps=len(validation_data),
epochs=100,
callbacks=[tf.keras.callbacks.EarlyStopping('val_loss', patience=3)]
)
Le modèle fait au maximum 47% de taux de succès sur les données de validation.
Comme nous observons un effet d'overfitting, malgré la présence de dropout, nous allons tester avec la data augmentation.
datagen = ImageDataGenerator(rescale=1/255,
validation_split=0.3,
zoom_range=0.1,
brightness_range=[0.7,1.2],
rotation_range=25,
horizontal_flip=True,
zca_whitening = True,
height_shift_range=0.05,
width_shift_range=0.05)
datagen_val = ImageDataGenerator(rescale=1/255,
validation_split=0.3)
train_data = datagen.flow_from_directory(batch_size=128,
directory=r'data',
shuffle=True,
seed = 1,
target_size=(224, 224),
color_mode="rgb",
subset="training",
class_mode='categorical'
)
validation_data = datagen_val.flow_from_directory(batch_size=128,
directory=r'data',
shuffle=True,
seed = 1,
target_size=(224, 224),
color_mode="rgb",
subset="validation",
class_mode='categorical')
Found 14458 images belonging to 120 classes. Found 6122 images belonging to 120 classes.
model = tf.keras.models.Sequential([
Conv2D(32, (3, 3), activation='relu', input_shape=(224, 224, 3), kernel_initializer=he_uniform()),
Conv2D(64, (3, 3), activation='relu', kernel_initializer=he_uniform()),
MaxPooling2D(2,2),
BatchNormalization(),
Conv2D(64, (3, 3), activation='relu', kernel_initializer=he_uniform()),
Conv2D(128, (3, 3), activation='relu', kernel_initializer=he_uniform()),
MaxPooling2D(2,2),
BatchNormalization(),
Conv2D(128, (3, 3), activation='relu', kernel_initializer=he_uniform()),
Conv2D(256, (3, 3), activation='relu', kernel_initializer=he_uniform()),
MaxPooling2D(2,2),
BatchNormalization(),
Conv2D(256, (3, 3), activation='relu', kernel_initializer=he_uniform()),
Conv2D(512, (3, 3), activation='relu', kernel_initializer=he_uniform()),
Conv2D(512, (3, 3), activation='relu', kernel_initializer=he_uniform()),
MaxPooling2D(2,2),
BatchNormalization(),
Conv2D(512, (3, 3), activation='relu', kernel_initializer=he_uniform()),
Conv2D(512, (3, 3), activation='relu', kernel_initializer=he_uniform()),
Conv2D(256, (3, 3), activation='relu', kernel_initializer=he_uniform()),
MaxPooling2D(2,2),
BatchNormalization(),
Flatten(),
Dense(512, activation='relu', kernel_initializer=he_uniform()),
Dropout(0.5),
Dense(256, activation='relu', kernel_initializer=he_uniform()),
Dropout(0.5),
Dense(len(races_list), activation='softmax')
])
model.compile(optimizer=Adam(learning_rate=0.0005, beta_1=0.95),
loss='categorical_crossentropy',
metrics=['accuracy'])
log_dir = "logs/fit/Deep_Model_DataAugmentation"
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
model.fit_generator(generator=train_data,
validation_data=validation_data,
steps_per_epoch=len(train_data),
validation_steps=len(validation_data),
epochs=100,
callbacks=[tf.keras.callbacks.EarlyStopping('val_loss', patience=3)]
)
Le modèle arrive maintenant a atteindre 48% de taux de succès.
Pour essayer d'améliorer nos résultats, nous allons utiliser le transfer learning. En commençant par le VGG16.
datagen = ImageDataGenerator(rescale=1/255,
validation_split=0.3,
zoom_range=0.1,
brightness_range=[0.7,1.2],
rotation_range=25,
horizontal_flip=True,
zca_whitening = True,
height_shift_range=0.05,
width_shift_range=0.05)
datagen_val = ImageDataGenerator(rescale=1/255,
validation_split=0.3)
train_data = datagen.flow_from_directory(batch_size=128,
directory=r'data',
shuffle=True,
seed = 1,
target_size=(224, 224),
color_mode="rgb",
subset="training",
class_mode='categorical'
)
validation_data = datagen_val.flow_from_directory(batch_size=128,
directory=r'data',
shuffle=True,
seed = 1,
target_size=(224, 224),
color_mode="rgb",
subset="validation",
class_mode='categorical')
Found 14458 images belonging to 120 classes. Found 6122 images belonging to 120 classes.
base_model = tf.keras.applications.VGG16(weights='imagenet', include_top = False, pooling='avg', input_shape=(224, 224, 3))
print(base_model.summary())
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/vgg16/vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5 58892288/58889256 [==============================] - 1s 0us/step Model: "vgg16" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_2 (InputLayer) [(None, 224, 224, 3)] 0 _________________________________________________________________ block1_conv1 (Conv2D) (None, 224, 224, 64) 1792 _________________________________________________________________ block1_conv2 (Conv2D) (None, 224, 224, 64) 36928 _________________________________________________________________ block1_pool (MaxPooling2D) (None, 112, 112, 64) 0 _________________________________________________________________ block2_conv1 (Conv2D) (None, 112, 112, 128) 73856 _________________________________________________________________ block2_conv2 (Conv2D) (None, 112, 112, 128) 147584 _________________________________________________________________ block2_pool (MaxPooling2D) (None, 56, 56, 128) 0 _________________________________________________________________ block3_conv1 (Conv2D) (None, 56, 56, 256) 295168 _________________________________________________________________ block3_conv2 (Conv2D) (None, 56, 56, 256) 590080 _________________________________________________________________ block3_conv3 (Conv2D) (None, 56, 56, 256) 590080 _________________________________________________________________ block3_pool (MaxPooling2D) (None, 28, 28, 256) 0 _________________________________________________________________ block4_conv1 (Conv2D) (None, 28, 28, 512) 1180160 _________________________________________________________________ block4_conv2 (Conv2D) (None, 28, 28, 512) 2359808 _________________________________________________________________ block4_conv3 (Conv2D) (None, 28, 28, 512) 2359808 _________________________________________________________________ block4_pool (MaxPooling2D) (None, 14, 14, 512) 0 _________________________________________________________________ block5_conv1 (Conv2D) (None, 14, 14, 512) 2359808 _________________________________________________________________ block5_conv2 (Conv2D) (None, 14, 14, 512) 2359808 _________________________________________________________________ block5_conv3 (Conv2D) (None, 14, 14, 512) 2359808 _________________________________________________________________ block5_pool (MaxPooling2D) (None, 7, 7, 512) 0 _________________________________________________________________ global_average_pooling2d (Gl (None, 512) 0 ================================================================= Total params: 14,714,688 Trainable params: 14,714,688 Non-trainable params: 0 _________________________________________________________________ None
Nous notons que le modèle est plus profond que le notre et possède plus de 14 millions de paramètres déjà entraînés.
base_model.trainable = False
model = tf.keras.models.Sequential([
base_model,
Dense(512, activation='relu', kernel_initializer=he_uniform()),
Dense(256, activation='relu', kernel_initializer=he_uniform()),
Dense(len(races_list), activation='softmax')
])
model.compile(optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'])
log_dir = "logs/fit/TL_Model_VGG16"
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
model.fit_generator(generator=train_data,
validation_data=validation_data,
steps_per_epoch=len(train_data),
validation_steps=len(validation_data),
epochs=100,
callbacks=[tf.keras.callbacks.EarlyStopping('val_loss', patience=3)]
)
Nous notons qu'avec 50 epochs, le modèle n'atteint que 43% de taux de succès. Cependant, nous remarquons qu'il converge au départ plus rapidement.
base_model.trainable = False
for l in base_model.layers[-4:]:
l.trainable = True
model = tf.keras.models.Sequential([
base_model,
Dense(128, activation='relu', kernel_initializer=he_uniform()),
Dense(len(races_list), activation='softmax')
])
model.compile(optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'])
log_dir = "logs/fit/TL_Model_FT_VGG16"
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
model.fit_generator(generator=train_data,
validation_data=validation_data,
steps_per_epoch=len(train_data),
validation_steps=len(validation_data),
epochs=100,
callbacks=[tf.keras.callbacks.EarlyStopping('val_loss', patience=3)]
)
En utilisant le fine tuning, donc entraînant les paramètres du modèle, les résultats obtenus ne sont pas meilleurs.
Prenons un autre modèle plus complexe.
base_model = tf.keras.applications.NASNetLarge(weights='imagenet', pooling='avg', include_top = False)
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/nasnet/NASNet-large-no-top.h5 343613440/343610240 [==============================] - 3s 0us/step
base_model.summary()
Model: "NASNet"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, 331, 331, 3) 0
__________________________________________________________________________________________________
stem_conv1 (Conv2D) (None, 165, 165, 96) 2592 input_1[0][0]
__________________________________________________________________________________________________
stem_bn1 (BatchNormalization) (None, 165, 165, 96) 384 stem_conv1[0][0]
__________________________________________________________________________________________________
activation (Activation) (None, 165, 165, 96) 0 stem_bn1[0][0]
__________________________________________________________________________________________________
reduction_conv_1_stem_1 (Conv2D (None, 165, 165, 42) 4032 activation[0][0]
__________________________________________________________________________________________________
reduction_bn_1_stem_1 (BatchNor (None, 165, 165, 42) 168 reduction_conv_1_stem_1[0][0]
__________________________________________________________________________________________________
activation_1 (Activation) (None, 165, 165, 42) 0 reduction_bn_1_stem_1[0][0]
__________________________________________________________________________________________________
activation_3 (Activation) (None, 165, 165, 96) 0 stem_bn1[0][0]
__________________________________________________________________________________________________
separable_conv_1_pad_reduction_ (None, 169, 169, 42) 0 activation_1[0][0]
__________________________________________________________________________________________________
separable_conv_1_pad_reduction_ (None, 171, 171, 96) 0 activation_3[0][0]
__________________________________________________________________________________________________
separable_conv_1_reduction_left (None, 83, 83, 42) 2814 separable_conv_1_pad_reduction_le
__________________________________________________________________________________________________
separable_conv_1_reduction_righ (None, 83, 83, 42) 8736 separable_conv_1_pad_reduction_ri
__________________________________________________________________________________________________
separable_conv_1_bn_reduction_l (None, 83, 83, 42) 168 separable_conv_1_reduction_left1_
__________________________________________________________________________________________________
separable_conv_1_bn_reduction_r (None, 83, 83, 42) 168 separable_conv_1_reduction_right1
__________________________________________________________________________________________________
activation_2 (Activation) (None, 83, 83, 42) 0 separable_conv_1_bn_reduction_lef
__________________________________________________________________________________________________
activation_4 (Activation) (None, 83, 83, 42) 0 separable_conv_1_bn_reduction_rig
__________________________________________________________________________________________________
separable_conv_2_reduction_left (None, 83, 83, 42) 2814 activation_2[0][0]
__________________________________________________________________________________________________
separable_conv_2_reduction_righ (None, 83, 83, 42) 3822 activation_4[0][0]
__________________________________________________________________________________________________
activation_5 (Activation) (None, 165, 165, 96) 0 stem_bn1[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_reduction_l (None, 83, 83, 42) 168 separable_conv_2_reduction_left1_
__________________________________________________________________________________________________
separable_conv_2_bn_reduction_r (None, 83, 83, 42) 168 separable_conv_2_reduction_right1
__________________________________________________________________________________________________
separable_conv_1_pad_reduction_ (None, 171, 171, 96) 0 activation_5[0][0]
__________________________________________________________________________________________________
activation_7 (Activation) (None, 165, 165, 96) 0 stem_bn1[0][0]
__________________________________________________________________________________________________
reduction_add_1_stem_1 (Add) (None, 83, 83, 42) 0 separable_conv_2_bn_reduction_lef
separable_conv_2_bn_reduction_rig
__________________________________________________________________________________________________
separable_conv_1_reduction_righ (None, 83, 83, 42) 8736 separable_conv_1_pad_reduction_ri
__________________________________________________________________________________________________
separable_conv_1_pad_reduction_ (None, 169, 169, 96) 0 activation_7[0][0]
__________________________________________________________________________________________________
activation_9 (Activation) (None, 83, 83, 42) 0 reduction_add_1_stem_1[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_reduction_r (None, 83, 83, 42) 168 separable_conv_1_reduction_right2
__________________________________________________________________________________________________
separable_conv_1_reduction_righ (None, 83, 83, 42) 6432 separable_conv_1_pad_reduction_ri
__________________________________________________________________________________________________
separable_conv_1_reduction_left (None, 83, 83, 42) 2142 activation_9[0][0]
__________________________________________________________________________________________________
activation_6 (Activation) (None, 83, 83, 42) 0 separable_conv_1_bn_reduction_rig
__________________________________________________________________________________________________
separable_conv_1_bn_reduction_r (None, 83, 83, 42) 168 separable_conv_1_reduction_right3
__________________________________________________________________________________________________
separable_conv_1_bn_reduction_l (None, 83, 83, 42) 168 separable_conv_1_reduction_left4_
__________________________________________________________________________________________________
reduction_pad_1_stem_1 (ZeroPad (None, 167, 167, 42) 0 reduction_bn_1_stem_1[0][0]
__________________________________________________________________________________________________
separable_conv_2_reduction_righ (None, 83, 83, 42) 3822 activation_6[0][0]
__________________________________________________________________________________________________
activation_8 (Activation) (None, 83, 83, 42) 0 separable_conv_1_bn_reduction_rig
__________________________________________________________________________________________________
activation_10 (Activation) (None, 83, 83, 42) 0 separable_conv_1_bn_reduction_lef
__________________________________________________________________________________________________
reduction_left2_stem_1 (MaxPool (None, 83, 83, 42) 0 reduction_pad_1_stem_1[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_reduction_r (None, 83, 83, 42) 168 separable_conv_2_reduction_right2
__________________________________________________________________________________________________
separable_conv_2_reduction_righ (None, 83, 83, 42) 2814 activation_8[0][0]
__________________________________________________________________________________________________
separable_conv_2_reduction_left (None, 83, 83, 42) 2142 activation_10[0][0]
__________________________________________________________________________________________________
adjust_relu_1_stem_2 (Activatio (None, 165, 165, 96) 0 stem_bn1[0][0]
__________________________________________________________________________________________________
reduction_add_2_stem_1 (Add) (None, 83, 83, 42) 0 reduction_left2_stem_1[0][0]
separable_conv_2_bn_reduction_rig
__________________________________________________________________________________________________
reduction_left3_stem_1 (Average (None, 83, 83, 42) 0 reduction_pad_1_stem_1[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_reduction_r (None, 83, 83, 42) 168 separable_conv_2_reduction_right3
__________________________________________________________________________________________________
reduction_left4_stem_1 (Average (None, 83, 83, 42) 0 reduction_add_1_stem_1[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_reduction_l (None, 83, 83, 42) 168 separable_conv_2_reduction_left4_
__________________________________________________________________________________________________
reduction_right5_stem_1 (MaxPoo (None, 83, 83, 42) 0 reduction_pad_1_stem_1[0][0]
__________________________________________________________________________________________________
zero_padding2d (ZeroPadding2D) (None, 166, 166, 96) 0 adjust_relu_1_stem_2[0][0]
__________________________________________________________________________________________________
reduction_add3_stem_1 (Add) (None, 83, 83, 42) 0 reduction_left3_stem_1[0][0]
separable_conv_2_bn_reduction_rig
__________________________________________________________________________________________________
add (Add) (None, 83, 83, 42) 0 reduction_add_2_stem_1[0][0]
reduction_left4_stem_1[0][0]
__________________________________________________________________________________________________
reduction_add4_stem_1 (Add) (None, 83, 83, 42) 0 separable_conv_2_bn_reduction_lef
reduction_right5_stem_1[0][0]
__________________________________________________________________________________________________
cropping2d (Cropping2D) (None, 165, 165, 96) 0 zero_padding2d[0][0]
__________________________________________________________________________________________________
reduction_concat_stem_1 (Concat (None, 83, 83, 168) 0 reduction_add_2_stem_1[0][0]
reduction_add3_stem_1[0][0]
add[0][0]
reduction_add4_stem_1[0][0]
__________________________________________________________________________________________________
adjust_avg_pool_1_stem_2 (Avera (None, 83, 83, 96) 0 adjust_relu_1_stem_2[0][0]
__________________________________________________________________________________________________
adjust_avg_pool_2_stem_2 (Avera (None, 83, 83, 96) 0 cropping2d[0][0]
__________________________________________________________________________________________________
activation_11 (Activation) (None, 83, 83, 168) 0 reduction_concat_stem_1[0][0]
__________________________________________________________________________________________________
adjust_conv_1_stem_2 (Conv2D) (None, 83, 83, 42) 4032 adjust_avg_pool_1_stem_2[0][0]
__________________________________________________________________________________________________
adjust_conv_2_stem_2 (Conv2D) (None, 83, 83, 42) 4032 adjust_avg_pool_2_stem_2[0][0]
__________________________________________________________________________________________________
reduction_conv_1_stem_2 (Conv2D (None, 83, 83, 84) 14112 activation_11[0][0]
__________________________________________________________________________________________________
concatenate (Concatenate) (None, 83, 83, 84) 0 adjust_conv_1_stem_2[0][0]
adjust_conv_2_stem_2[0][0]
__________________________________________________________________________________________________
reduction_bn_1_stem_2 (BatchNor (None, 83, 83, 84) 336 reduction_conv_1_stem_2[0][0]
__________________________________________________________________________________________________
adjust_bn_stem_2 (BatchNormaliz (None, 83, 83, 84) 336 concatenate[0][0]
__________________________________________________________________________________________________
activation_12 (Activation) (None, 83, 83, 84) 0 reduction_bn_1_stem_2[0][0]
__________________________________________________________________________________________________
activation_14 (Activation) (None, 83, 83, 84) 0 adjust_bn_stem_2[0][0]
__________________________________________________________________________________________________
separable_conv_1_pad_reduction_ (None, 87, 87, 84) 0 activation_12[0][0]
__________________________________________________________________________________________________
separable_conv_1_pad_reduction_ (None, 89, 89, 84) 0 activation_14[0][0]
__________________________________________________________________________________________________
separable_conv_1_reduction_left (None, 42, 42, 84) 9156 separable_conv_1_pad_reduction_le
__________________________________________________________________________________________________
separable_conv_1_reduction_righ (None, 42, 42, 84) 11172 separable_conv_1_pad_reduction_ri
__________________________________________________________________________________________________
separable_conv_1_bn_reduction_l (None, 42, 42, 84) 336 separable_conv_1_reduction_left1_
__________________________________________________________________________________________________
separable_conv_1_bn_reduction_r (None, 42, 42, 84) 336 separable_conv_1_reduction_right1
__________________________________________________________________________________________________
activation_13 (Activation) (None, 42, 42, 84) 0 separable_conv_1_bn_reduction_lef
__________________________________________________________________________________________________
activation_15 (Activation) (None, 42, 42, 84) 0 separable_conv_1_bn_reduction_rig
__________________________________________________________________________________________________
separable_conv_2_reduction_left (None, 42, 42, 84) 9156 activation_13[0][0]
__________________________________________________________________________________________________
separable_conv_2_reduction_righ (None, 42, 42, 84) 11172 activation_15[0][0]
__________________________________________________________________________________________________
activation_16 (Activation) (None, 83, 83, 84) 0 adjust_bn_stem_2[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_reduction_l (None, 42, 42, 84) 336 separable_conv_2_reduction_left1_
__________________________________________________________________________________________________
separable_conv_2_bn_reduction_r (None, 42, 42, 84) 336 separable_conv_2_reduction_right1
__________________________________________________________________________________________________
separable_conv_1_pad_reduction_ (None, 89, 89, 84) 0 activation_16[0][0]
__________________________________________________________________________________________________
activation_18 (Activation) (None, 83, 83, 84) 0 adjust_bn_stem_2[0][0]
__________________________________________________________________________________________________
reduction_add_1_stem_2 (Add) (None, 42, 42, 84) 0 separable_conv_2_bn_reduction_lef
separable_conv_2_bn_reduction_rig
__________________________________________________________________________________________________
separable_conv_1_reduction_righ (None, 42, 42, 84) 11172 separable_conv_1_pad_reduction_ri
__________________________________________________________________________________________________
separable_conv_1_pad_reduction_ (None, 87, 87, 84) 0 activation_18[0][0]
__________________________________________________________________________________________________
activation_20 (Activation) (None, 42, 42, 84) 0 reduction_add_1_stem_2[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_reduction_r (None, 42, 42, 84) 336 separable_conv_1_reduction_right2
__________________________________________________________________________________________________
separable_conv_1_reduction_righ (None, 42, 42, 84) 9156 separable_conv_1_pad_reduction_ri
__________________________________________________________________________________________________
separable_conv_1_reduction_left (None, 42, 42, 84) 7812 activation_20[0][0]
__________________________________________________________________________________________________
activation_17 (Activation) (None, 42, 42, 84) 0 separable_conv_1_bn_reduction_rig
__________________________________________________________________________________________________
separable_conv_1_bn_reduction_r (None, 42, 42, 84) 336 separable_conv_1_reduction_right3
__________________________________________________________________________________________________
separable_conv_1_bn_reduction_l (None, 42, 42, 84) 336 separable_conv_1_reduction_left4_
__________________________________________________________________________________________________
reduction_pad_1_stem_2 (ZeroPad (None, 85, 85, 84) 0 reduction_bn_1_stem_2[0][0]
__________________________________________________________________________________________________
separable_conv_2_reduction_righ (None, 42, 42, 84) 11172 activation_17[0][0]
__________________________________________________________________________________________________
activation_19 (Activation) (None, 42, 42, 84) 0 separable_conv_1_bn_reduction_rig
__________________________________________________________________________________________________
activation_21 (Activation) (None, 42, 42, 84) 0 separable_conv_1_bn_reduction_lef
__________________________________________________________________________________________________
reduction_left2_stem_2 (MaxPool (None, 42, 42, 84) 0 reduction_pad_1_stem_2[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_reduction_r (None, 42, 42, 84) 336 separable_conv_2_reduction_right2
__________________________________________________________________________________________________
separable_conv_2_reduction_righ (None, 42, 42, 84) 9156 activation_19[0][0]
__________________________________________________________________________________________________
separable_conv_2_reduction_left (None, 42, 42, 84) 7812 activation_21[0][0]
__________________________________________________________________________________________________
adjust_relu_1_0 (Activation) (None, 83, 83, 168) 0 reduction_concat_stem_1[0][0]
__________________________________________________________________________________________________
reduction_add_2_stem_2 (Add) (None, 42, 42, 84) 0 reduction_left2_stem_2[0][0]
separable_conv_2_bn_reduction_rig
__________________________________________________________________________________________________
reduction_left3_stem_2 (Average (None, 42, 42, 84) 0 reduction_pad_1_stem_2[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_reduction_r (None, 42, 42, 84) 336 separable_conv_2_reduction_right3
__________________________________________________________________________________________________
reduction_left4_stem_2 (Average (None, 42, 42, 84) 0 reduction_add_1_stem_2[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_reduction_l (None, 42, 42, 84) 336 separable_conv_2_reduction_left4_
__________________________________________________________________________________________________
reduction_right5_stem_2 (MaxPoo (None, 42, 42, 84) 0 reduction_pad_1_stem_2[0][0]
__________________________________________________________________________________________________
zero_padding2d_1 (ZeroPadding2D (None, 84, 84, 168) 0 adjust_relu_1_0[0][0]
__________________________________________________________________________________________________
reduction_add3_stem_2 (Add) (None, 42, 42, 84) 0 reduction_left3_stem_2[0][0]
separable_conv_2_bn_reduction_rig
__________________________________________________________________________________________________
add_1 (Add) (None, 42, 42, 84) 0 reduction_add_2_stem_2[0][0]
reduction_left4_stem_2[0][0]
__________________________________________________________________________________________________
reduction_add4_stem_2 (Add) (None, 42, 42, 84) 0 separable_conv_2_bn_reduction_lef
reduction_right5_stem_2[0][0]
__________________________________________________________________________________________________
cropping2d_1 (Cropping2D) (None, 83, 83, 168) 0 zero_padding2d_1[0][0]
__________________________________________________________________________________________________
reduction_concat_stem_2 (Concat (None, 42, 42, 336) 0 reduction_add_2_stem_2[0][0]
reduction_add3_stem_2[0][0]
add_1[0][0]
reduction_add4_stem_2[0][0]
__________________________________________________________________________________________________
adjust_avg_pool_1_0 (AveragePoo (None, 42, 42, 168) 0 adjust_relu_1_0[0][0]
__________________________________________________________________________________________________
adjust_avg_pool_2_0 (AveragePoo (None, 42, 42, 168) 0 cropping2d_1[0][0]
__________________________________________________________________________________________________
adjust_conv_1_0 (Conv2D) (None, 42, 42, 84) 14112 adjust_avg_pool_1_0[0][0]
__________________________________________________________________________________________________
adjust_conv_2_0 (Conv2D) (None, 42, 42, 84) 14112 adjust_avg_pool_2_0[0][0]
__________________________________________________________________________________________________
activation_22 (Activation) (None, 42, 42, 336) 0 reduction_concat_stem_2[0][0]
__________________________________________________________________________________________________
concatenate_1 (Concatenate) (None, 42, 42, 168) 0 adjust_conv_1_0[0][0]
adjust_conv_2_0[0][0]
__________________________________________________________________________________________________
normal_conv_1_0 (Conv2D) (None, 42, 42, 168) 56448 activation_22[0][0]
__________________________________________________________________________________________________
adjust_bn_0 (BatchNormalization (None, 42, 42, 168) 672 concatenate_1[0][0]
__________________________________________________________________________________________________
normal_bn_1_0 (BatchNormalizati (None, 42, 42, 168) 672 normal_conv_1_0[0][0]
__________________________________________________________________________________________________
activation_23 (Activation) (None, 42, 42, 168) 0 normal_bn_1_0[0][0]
__________________________________________________________________________________________________
activation_25 (Activation) (None, 42, 42, 168) 0 adjust_bn_0[0][0]
__________________________________________________________________________________________________
activation_27 (Activation) (None, 42, 42, 168) 0 adjust_bn_0[0][0]
__________________________________________________________________________________________________
activation_29 (Activation) (None, 42, 42, 168) 0 adjust_bn_0[0][0]
__________________________________________________________________________________________________
activation_31 (Activation) (None, 42, 42, 168) 0 normal_bn_1_0[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left1_0 (None, 42, 42, 168) 32424 activation_23[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right1_ (None, 42, 42, 168) 29736 activation_25[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left2_0 (None, 42, 42, 168) 32424 activation_27[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right2_ (None, 42, 42, 168) 29736 activation_29[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left5_0 (None, 42, 42, 168) 29736 activation_31[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 42, 42, 168) 672 separable_conv_1_normal_left1_0[0
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_1_normal_right1_0[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 42, 42, 168) 672 separable_conv_1_normal_left2_0[0
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_1_normal_right2_0[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 42, 42, 168) 672 separable_conv_1_normal_left5_0[0
__________________________________________________________________________________________________
activation_24 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_left1_
__________________________________________________________________________________________________
activation_26 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_right1
__________________________________________________________________________________________________
activation_28 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_left2_
__________________________________________________________________________________________________
activation_30 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_right2
__________________________________________________________________________________________________
activation_32 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_left5_
__________________________________________________________________________________________________
separable_conv_2_normal_left1_0 (None, 42, 42, 168) 32424 activation_24[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right1_ (None, 42, 42, 168) 29736 activation_26[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left2_0 (None, 42, 42, 168) 32424 activation_28[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right2_ (None, 42, 42, 168) 29736 activation_30[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left5_0 (None, 42, 42, 168) 29736 activation_32[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 42, 42, 168) 672 separable_conv_2_normal_left1_0[0
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_2_normal_right1_0[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 42, 42, 168) 672 separable_conv_2_normal_left2_0[0
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_2_normal_right2_0[
__________________________________________________________________________________________________
normal_left3_0 (AveragePooling2 (None, 42, 42, 168) 0 normal_bn_1_0[0][0]
__________________________________________________________________________________________________
normal_left4_0 (AveragePooling2 (None, 42, 42, 168) 0 adjust_bn_0[0][0]
__________________________________________________________________________________________________
normal_right4_0 (AveragePooling (None, 42, 42, 168) 0 adjust_bn_0[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 42, 42, 168) 672 separable_conv_2_normal_left5_0[0
__________________________________________________________________________________________________
normal_add_1_0 (Add) (None, 42, 42, 168) 0 separable_conv_2_bn_normal_left1_
separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________
normal_add_2_0 (Add) (None, 42, 42, 168) 0 separable_conv_2_bn_normal_left2_
separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________
normal_add_3_0 (Add) (None, 42, 42, 168) 0 normal_left3_0[0][0]
adjust_bn_0[0][0]
__________________________________________________________________________________________________
normal_add_4_0 (Add) (None, 42, 42, 168) 0 normal_left4_0[0][0]
normal_right4_0[0][0]
__________________________________________________________________________________________________
normal_add_5_0 (Add) (None, 42, 42, 168) 0 separable_conv_2_bn_normal_left5_
normal_bn_1_0[0][0]
__________________________________________________________________________________________________
normal_concat_0 (Concatenate) (None, 42, 42, 1008) 0 adjust_bn_0[0][0]
normal_add_1_0[0][0]
normal_add_2_0[0][0]
normal_add_3_0[0][0]
normal_add_4_0[0][0]
normal_add_5_0[0][0]
__________________________________________________________________________________________________
activation_33 (Activation) (None, 42, 42, 336) 0 reduction_concat_stem_2[0][0]
__________________________________________________________________________________________________
activation_34 (Activation) (None, 42, 42, 1008) 0 normal_concat_0[0][0]
__________________________________________________________________________________________________
adjust_conv_projection_1 (Conv2 (None, 42, 42, 168) 56448 activation_33[0][0]
__________________________________________________________________________________________________
normal_conv_1_1 (Conv2D) (None, 42, 42, 168) 169344 activation_34[0][0]
__________________________________________________________________________________________________
adjust_bn_1 (BatchNormalization (None, 42, 42, 168) 672 adjust_conv_projection_1[0][0]
__________________________________________________________________________________________________
normal_bn_1_1 (BatchNormalizati (None, 42, 42, 168) 672 normal_conv_1_1[0][0]
__________________________________________________________________________________________________
activation_35 (Activation) (None, 42, 42, 168) 0 normal_bn_1_1[0][0]
__________________________________________________________________________________________________
activation_37 (Activation) (None, 42, 42, 168) 0 adjust_bn_1[0][0]
__________________________________________________________________________________________________
activation_39 (Activation) (None, 42, 42, 168) 0 adjust_bn_1[0][0]
__________________________________________________________________________________________________
activation_41 (Activation) (None, 42, 42, 168) 0 adjust_bn_1[0][0]
__________________________________________________________________________________________________
activation_43 (Activation) (None, 42, 42, 168) 0 normal_bn_1_1[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left1_1 (None, 42, 42, 168) 32424 activation_35[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right1_ (None, 42, 42, 168) 29736 activation_37[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left2_1 (None, 42, 42, 168) 32424 activation_39[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right2_ (None, 42, 42, 168) 29736 activation_41[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left5_1 (None, 42, 42, 168) 29736 activation_43[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 42, 42, 168) 672 separable_conv_1_normal_left1_1[0
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_1_normal_right1_1[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 42, 42, 168) 672 separable_conv_1_normal_left2_1[0
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_1_normal_right2_1[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 42, 42, 168) 672 separable_conv_1_normal_left5_1[0
__________________________________________________________________________________________________
activation_36 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_left1_
__________________________________________________________________________________________________
activation_38 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_right1
__________________________________________________________________________________________________
activation_40 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_left2_
__________________________________________________________________________________________________
activation_42 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_right2
__________________________________________________________________________________________________
activation_44 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_left5_
__________________________________________________________________________________________________
separable_conv_2_normal_left1_1 (None, 42, 42, 168) 32424 activation_36[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right1_ (None, 42, 42, 168) 29736 activation_38[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left2_1 (None, 42, 42, 168) 32424 activation_40[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right2_ (None, 42, 42, 168) 29736 activation_42[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left5_1 (None, 42, 42, 168) 29736 activation_44[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 42, 42, 168) 672 separable_conv_2_normal_left1_1[0
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_2_normal_right1_1[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 42, 42, 168) 672 separable_conv_2_normal_left2_1[0
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_2_normal_right2_1[
__________________________________________________________________________________________________
normal_left3_1 (AveragePooling2 (None, 42, 42, 168) 0 normal_bn_1_1[0][0]
__________________________________________________________________________________________________
normal_left4_1 (AveragePooling2 (None, 42, 42, 168) 0 adjust_bn_1[0][0]
__________________________________________________________________________________________________
normal_right4_1 (AveragePooling (None, 42, 42, 168) 0 adjust_bn_1[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 42, 42, 168) 672 separable_conv_2_normal_left5_1[0
__________________________________________________________________________________________________
normal_add_1_1 (Add) (None, 42, 42, 168) 0 separable_conv_2_bn_normal_left1_
separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________
normal_add_2_1 (Add) (None, 42, 42, 168) 0 separable_conv_2_bn_normal_left2_
separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________
normal_add_3_1 (Add) (None, 42, 42, 168) 0 normal_left3_1[0][0]
adjust_bn_1[0][0]
__________________________________________________________________________________________________
normal_add_4_1 (Add) (None, 42, 42, 168) 0 normal_left4_1[0][0]
normal_right4_1[0][0]
__________________________________________________________________________________________________
normal_add_5_1 (Add) (None, 42, 42, 168) 0 separable_conv_2_bn_normal_left5_
normal_bn_1_1[0][0]
__________________________________________________________________________________________________
normal_concat_1 (Concatenate) (None, 42, 42, 1008) 0 adjust_bn_1[0][0]
normal_add_1_1[0][0]
normal_add_2_1[0][0]
normal_add_3_1[0][0]
normal_add_4_1[0][0]
normal_add_5_1[0][0]
__________________________________________________________________________________________________
activation_45 (Activation) (None, 42, 42, 1008) 0 normal_concat_0[0][0]
__________________________________________________________________________________________________
activation_46 (Activation) (None, 42, 42, 1008) 0 normal_concat_1[0][0]
__________________________________________________________________________________________________
adjust_conv_projection_2 (Conv2 (None, 42, 42, 168) 169344 activation_45[0][0]
__________________________________________________________________________________________________
normal_conv_1_2 (Conv2D) (None, 42, 42, 168) 169344 activation_46[0][0]
__________________________________________________________________________________________________
adjust_bn_2 (BatchNormalization (None, 42, 42, 168) 672 adjust_conv_projection_2[0][0]
__________________________________________________________________________________________________
normal_bn_1_2 (BatchNormalizati (None, 42, 42, 168) 672 normal_conv_1_2[0][0]
__________________________________________________________________________________________________
activation_47 (Activation) (None, 42, 42, 168) 0 normal_bn_1_2[0][0]
__________________________________________________________________________________________________
activation_49 (Activation) (None, 42, 42, 168) 0 adjust_bn_2[0][0]
__________________________________________________________________________________________________
activation_51 (Activation) (None, 42, 42, 168) 0 adjust_bn_2[0][0]
__________________________________________________________________________________________________
activation_53 (Activation) (None, 42, 42, 168) 0 adjust_bn_2[0][0]
__________________________________________________________________________________________________
activation_55 (Activation) (None, 42, 42, 168) 0 normal_bn_1_2[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left1_2 (None, 42, 42, 168) 32424 activation_47[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right1_ (None, 42, 42, 168) 29736 activation_49[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left2_2 (None, 42, 42, 168) 32424 activation_51[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right2_ (None, 42, 42, 168) 29736 activation_53[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left5_2 (None, 42, 42, 168) 29736 activation_55[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 42, 42, 168) 672 separable_conv_1_normal_left1_2[0
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_1_normal_right1_2[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 42, 42, 168) 672 separable_conv_1_normal_left2_2[0
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_1_normal_right2_2[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 42, 42, 168) 672 separable_conv_1_normal_left5_2[0
__________________________________________________________________________________________________
activation_48 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_left1_
__________________________________________________________________________________________________
activation_50 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_right1
__________________________________________________________________________________________________
activation_52 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_left2_
__________________________________________________________________________________________________
activation_54 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_right2
__________________________________________________________________________________________________
activation_56 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_left5_
__________________________________________________________________________________________________
separable_conv_2_normal_left1_2 (None, 42, 42, 168) 32424 activation_48[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right1_ (None, 42, 42, 168) 29736 activation_50[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left2_2 (None, 42, 42, 168) 32424 activation_52[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right2_ (None, 42, 42, 168) 29736 activation_54[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left5_2 (None, 42, 42, 168) 29736 activation_56[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 42, 42, 168) 672 separable_conv_2_normal_left1_2[0
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_2_normal_right1_2[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 42, 42, 168) 672 separable_conv_2_normal_left2_2[0
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_2_normal_right2_2[
__________________________________________________________________________________________________
normal_left3_2 (AveragePooling2 (None, 42, 42, 168) 0 normal_bn_1_2[0][0]
__________________________________________________________________________________________________
normal_left4_2 (AveragePooling2 (None, 42, 42, 168) 0 adjust_bn_2[0][0]
__________________________________________________________________________________________________
normal_right4_2 (AveragePooling (None, 42, 42, 168) 0 adjust_bn_2[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 42, 42, 168) 672 separable_conv_2_normal_left5_2[0
__________________________________________________________________________________________________
normal_add_1_2 (Add) (None, 42, 42, 168) 0 separable_conv_2_bn_normal_left1_
separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________
normal_add_2_2 (Add) (None, 42, 42, 168) 0 separable_conv_2_bn_normal_left2_
separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________
normal_add_3_2 (Add) (None, 42, 42, 168) 0 normal_left3_2[0][0]
adjust_bn_2[0][0]
__________________________________________________________________________________________________
normal_add_4_2 (Add) (None, 42, 42, 168) 0 normal_left4_2[0][0]
normal_right4_2[0][0]
__________________________________________________________________________________________________
normal_add_5_2 (Add) (None, 42, 42, 168) 0 separable_conv_2_bn_normal_left5_
normal_bn_1_2[0][0]
__________________________________________________________________________________________________
normal_concat_2 (Concatenate) (None, 42, 42, 1008) 0 adjust_bn_2[0][0]
normal_add_1_2[0][0]
normal_add_2_2[0][0]
normal_add_3_2[0][0]
normal_add_4_2[0][0]
normal_add_5_2[0][0]
__________________________________________________________________________________________________
activation_57 (Activation) (None, 42, 42, 1008) 0 normal_concat_1[0][0]
__________________________________________________________________________________________________
activation_58 (Activation) (None, 42, 42, 1008) 0 normal_concat_2[0][0]
__________________________________________________________________________________________________
adjust_conv_projection_3 (Conv2 (None, 42, 42, 168) 169344 activation_57[0][0]
__________________________________________________________________________________________________
normal_conv_1_3 (Conv2D) (None, 42, 42, 168) 169344 activation_58[0][0]
__________________________________________________________________________________________________
adjust_bn_3 (BatchNormalization (None, 42, 42, 168) 672 adjust_conv_projection_3[0][0]
__________________________________________________________________________________________________
normal_bn_1_3 (BatchNormalizati (None, 42, 42, 168) 672 normal_conv_1_3[0][0]
__________________________________________________________________________________________________
activation_59 (Activation) (None, 42, 42, 168) 0 normal_bn_1_3[0][0]
__________________________________________________________________________________________________
activation_61 (Activation) (None, 42, 42, 168) 0 adjust_bn_3[0][0]
__________________________________________________________________________________________________
activation_63 (Activation) (None, 42, 42, 168) 0 adjust_bn_3[0][0]
__________________________________________________________________________________________________
activation_65 (Activation) (None, 42, 42, 168) 0 adjust_bn_3[0][0]
__________________________________________________________________________________________________
activation_67 (Activation) (None, 42, 42, 168) 0 normal_bn_1_3[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left1_3 (None, 42, 42, 168) 32424 activation_59[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right1_ (None, 42, 42, 168) 29736 activation_61[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left2_3 (None, 42, 42, 168) 32424 activation_63[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right2_ (None, 42, 42, 168) 29736 activation_65[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left5_3 (None, 42, 42, 168) 29736 activation_67[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 42, 42, 168) 672 separable_conv_1_normal_left1_3[0
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_1_normal_right1_3[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 42, 42, 168) 672 separable_conv_1_normal_left2_3[0
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_1_normal_right2_3[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 42, 42, 168) 672 separable_conv_1_normal_left5_3[0
__________________________________________________________________________________________________
activation_60 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_left1_
__________________________________________________________________________________________________
activation_62 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_right1
__________________________________________________________________________________________________
activation_64 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_left2_
__________________________________________________________________________________________________
activation_66 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_right2
__________________________________________________________________________________________________
activation_68 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_left5_
__________________________________________________________________________________________________
separable_conv_2_normal_left1_3 (None, 42, 42, 168) 32424 activation_60[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right1_ (None, 42, 42, 168) 29736 activation_62[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left2_3 (None, 42, 42, 168) 32424 activation_64[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right2_ (None, 42, 42, 168) 29736 activation_66[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left5_3 (None, 42, 42, 168) 29736 activation_68[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 42, 42, 168) 672 separable_conv_2_normal_left1_3[0
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_2_normal_right1_3[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 42, 42, 168) 672 separable_conv_2_normal_left2_3[0
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_2_normal_right2_3[
__________________________________________________________________________________________________
normal_left3_3 (AveragePooling2 (None, 42, 42, 168) 0 normal_bn_1_3[0][0]
__________________________________________________________________________________________________
normal_left4_3 (AveragePooling2 (None, 42, 42, 168) 0 adjust_bn_3[0][0]
__________________________________________________________________________________________________
normal_right4_3 (AveragePooling (None, 42, 42, 168) 0 adjust_bn_3[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 42, 42, 168) 672 separable_conv_2_normal_left5_3[0
__________________________________________________________________________________________________
normal_add_1_3 (Add) (None, 42, 42, 168) 0 separable_conv_2_bn_normal_left1_
separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________
normal_add_2_3 (Add) (None, 42, 42, 168) 0 separable_conv_2_bn_normal_left2_
separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________
normal_add_3_3 (Add) (None, 42, 42, 168) 0 normal_left3_3[0][0]
adjust_bn_3[0][0]
__________________________________________________________________________________________________
normal_add_4_3 (Add) (None, 42, 42, 168) 0 normal_left4_3[0][0]
normal_right4_3[0][0]
__________________________________________________________________________________________________
normal_add_5_3 (Add) (None, 42, 42, 168) 0 separable_conv_2_bn_normal_left5_
normal_bn_1_3[0][0]
__________________________________________________________________________________________________
normal_concat_3 (Concatenate) (None, 42, 42, 1008) 0 adjust_bn_3[0][0]
normal_add_1_3[0][0]
normal_add_2_3[0][0]
normal_add_3_3[0][0]
normal_add_4_3[0][0]
normal_add_5_3[0][0]
__________________________________________________________________________________________________
activation_69 (Activation) (None, 42, 42, 1008) 0 normal_concat_2[0][0]
__________________________________________________________________________________________________
activation_70 (Activation) (None, 42, 42, 1008) 0 normal_concat_3[0][0]
__________________________________________________________________________________________________
adjust_conv_projection_4 (Conv2 (None, 42, 42, 168) 169344 activation_69[0][0]
__________________________________________________________________________________________________
normal_conv_1_4 (Conv2D) (None, 42, 42, 168) 169344 activation_70[0][0]
__________________________________________________________________________________________________
adjust_bn_4 (BatchNormalization (None, 42, 42, 168) 672 adjust_conv_projection_4[0][0]
__________________________________________________________________________________________________
normal_bn_1_4 (BatchNormalizati (None, 42, 42, 168) 672 normal_conv_1_4[0][0]
__________________________________________________________________________________________________
activation_71 (Activation) (None, 42, 42, 168) 0 normal_bn_1_4[0][0]
__________________________________________________________________________________________________
activation_73 (Activation) (None, 42, 42, 168) 0 adjust_bn_4[0][0]
__________________________________________________________________________________________________
activation_75 (Activation) (None, 42, 42, 168) 0 adjust_bn_4[0][0]
__________________________________________________________________________________________________
activation_77 (Activation) (None, 42, 42, 168) 0 adjust_bn_4[0][0]
__________________________________________________________________________________________________
activation_79 (Activation) (None, 42, 42, 168) 0 normal_bn_1_4[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left1_4 (None, 42, 42, 168) 32424 activation_71[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right1_ (None, 42, 42, 168) 29736 activation_73[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left2_4 (None, 42, 42, 168) 32424 activation_75[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right2_ (None, 42, 42, 168) 29736 activation_77[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left5_4 (None, 42, 42, 168) 29736 activation_79[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 42, 42, 168) 672 separable_conv_1_normal_left1_4[0
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_1_normal_right1_4[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 42, 42, 168) 672 separable_conv_1_normal_left2_4[0
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_1_normal_right2_4[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 42, 42, 168) 672 separable_conv_1_normal_left5_4[0
__________________________________________________________________________________________________
activation_72 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_left1_
__________________________________________________________________________________________________
activation_74 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_right1
__________________________________________________________________________________________________
activation_76 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_left2_
__________________________________________________________________________________________________
activation_78 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_right2
__________________________________________________________________________________________________
activation_80 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_left5_
__________________________________________________________________________________________________
separable_conv_2_normal_left1_4 (None, 42, 42, 168) 32424 activation_72[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right1_ (None, 42, 42, 168) 29736 activation_74[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left2_4 (None, 42, 42, 168) 32424 activation_76[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right2_ (None, 42, 42, 168) 29736 activation_78[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left5_4 (None, 42, 42, 168) 29736 activation_80[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 42, 42, 168) 672 separable_conv_2_normal_left1_4[0
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_2_normal_right1_4[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 42, 42, 168) 672 separable_conv_2_normal_left2_4[0
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_2_normal_right2_4[
__________________________________________________________________________________________________
normal_left3_4 (AveragePooling2 (None, 42, 42, 168) 0 normal_bn_1_4[0][0]
__________________________________________________________________________________________________
normal_left4_4 (AveragePooling2 (None, 42, 42, 168) 0 adjust_bn_4[0][0]
__________________________________________________________________________________________________
normal_right4_4 (AveragePooling (None, 42, 42, 168) 0 adjust_bn_4[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 42, 42, 168) 672 separable_conv_2_normal_left5_4[0
__________________________________________________________________________________________________
normal_add_1_4 (Add) (None, 42, 42, 168) 0 separable_conv_2_bn_normal_left1_
separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________
normal_add_2_4 (Add) (None, 42, 42, 168) 0 separable_conv_2_bn_normal_left2_
separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________
normal_add_3_4 (Add) (None, 42, 42, 168) 0 normal_left3_4[0][0]
adjust_bn_4[0][0]
__________________________________________________________________________________________________
normal_add_4_4 (Add) (None, 42, 42, 168) 0 normal_left4_4[0][0]
normal_right4_4[0][0]
__________________________________________________________________________________________________
normal_add_5_4 (Add) (None, 42, 42, 168) 0 separable_conv_2_bn_normal_left5_
normal_bn_1_4[0][0]
__________________________________________________________________________________________________
normal_concat_4 (Concatenate) (None, 42, 42, 1008) 0 adjust_bn_4[0][0]
normal_add_1_4[0][0]
normal_add_2_4[0][0]
normal_add_3_4[0][0]
normal_add_4_4[0][0]
normal_add_5_4[0][0]
__________________________________________________________________________________________________
activation_81 (Activation) (None, 42, 42, 1008) 0 normal_concat_3[0][0]
__________________________________________________________________________________________________
activation_82 (Activation) (None, 42, 42, 1008) 0 normal_concat_4[0][0]
__________________________________________________________________________________________________
adjust_conv_projection_5 (Conv2 (None, 42, 42, 168) 169344 activation_81[0][0]
__________________________________________________________________________________________________
normal_conv_1_5 (Conv2D) (None, 42, 42, 168) 169344 activation_82[0][0]
__________________________________________________________________________________________________
adjust_bn_5 (BatchNormalization (None, 42, 42, 168) 672 adjust_conv_projection_5[0][0]
__________________________________________________________________________________________________
normal_bn_1_5 (BatchNormalizati (None, 42, 42, 168) 672 normal_conv_1_5[0][0]
__________________________________________________________________________________________________
activation_83 (Activation) (None, 42, 42, 168) 0 normal_bn_1_5[0][0]
__________________________________________________________________________________________________
activation_85 (Activation) (None, 42, 42, 168) 0 adjust_bn_5[0][0]
__________________________________________________________________________________________________
activation_87 (Activation) (None, 42, 42, 168) 0 adjust_bn_5[0][0]
__________________________________________________________________________________________________
activation_89 (Activation) (None, 42, 42, 168) 0 adjust_bn_5[0][0]
__________________________________________________________________________________________________
activation_91 (Activation) (None, 42, 42, 168) 0 normal_bn_1_5[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left1_5 (None, 42, 42, 168) 32424 activation_83[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right1_ (None, 42, 42, 168) 29736 activation_85[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left2_5 (None, 42, 42, 168) 32424 activation_87[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right2_ (None, 42, 42, 168) 29736 activation_89[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left5_5 (None, 42, 42, 168) 29736 activation_91[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 42, 42, 168) 672 separable_conv_1_normal_left1_5[0
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_1_normal_right1_5[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 42, 42, 168) 672 separable_conv_1_normal_left2_5[0
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_1_normal_right2_5[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 42, 42, 168) 672 separable_conv_1_normal_left5_5[0
__________________________________________________________________________________________________
activation_84 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_left1_
__________________________________________________________________________________________________
activation_86 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_right1
__________________________________________________________________________________________________
activation_88 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_left2_
__________________________________________________________________________________________________
activation_90 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_right2
__________________________________________________________________________________________________
activation_92 (Activation) (None, 42, 42, 168) 0 separable_conv_1_bn_normal_left5_
__________________________________________________________________________________________________
separable_conv_2_normal_left1_5 (None, 42, 42, 168) 32424 activation_84[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right1_ (None, 42, 42, 168) 29736 activation_86[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left2_5 (None, 42, 42, 168) 32424 activation_88[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right2_ (None, 42, 42, 168) 29736 activation_90[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left5_5 (None, 42, 42, 168) 29736 activation_92[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 42, 42, 168) 672 separable_conv_2_normal_left1_5[0
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_2_normal_right1_5[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 42, 42, 168) 672 separable_conv_2_normal_left2_5[0
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 42, 42, 168) 672 separable_conv_2_normal_right2_5[
__________________________________________________________________________________________________
normal_left3_5 (AveragePooling2 (None, 42, 42, 168) 0 normal_bn_1_5[0][0]
__________________________________________________________________________________________________
normal_left4_5 (AveragePooling2 (None, 42, 42, 168) 0 adjust_bn_5[0][0]
__________________________________________________________________________________________________
normal_right4_5 (AveragePooling (None, 42, 42, 168) 0 adjust_bn_5[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 42, 42, 168) 672 separable_conv_2_normal_left5_5[0
__________________________________________________________________________________________________
normal_add_1_5 (Add) (None, 42, 42, 168) 0 separable_conv_2_bn_normal_left1_
separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________
normal_add_2_5 (Add) (None, 42, 42, 168) 0 separable_conv_2_bn_normal_left2_
separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________
normal_add_3_5 (Add) (None, 42, 42, 168) 0 normal_left3_5[0][0]
adjust_bn_5[0][0]
__________________________________________________________________________________________________
normal_add_4_5 (Add) (None, 42, 42, 168) 0 normal_left4_5[0][0]
normal_right4_5[0][0]
__________________________________________________________________________________________________
normal_add_5_5 (Add) (None, 42, 42, 168) 0 separable_conv_2_bn_normal_left5_
normal_bn_1_5[0][0]
__________________________________________________________________________________________________
normal_concat_5 (Concatenate) (None, 42, 42, 1008) 0 adjust_bn_5[0][0]
normal_add_1_5[0][0]
normal_add_2_5[0][0]
normal_add_3_5[0][0]
normal_add_4_5[0][0]
normal_add_5_5[0][0]
__________________________________________________________________________________________________
activation_94 (Activation) (None, 42, 42, 1008) 0 normal_concat_5[0][0]
__________________________________________________________________________________________________
activation_93 (Activation) (None, 42, 42, 1008) 0 normal_concat_4[0][0]
__________________________________________________________________________________________________
reduction_conv_1_reduce_6 (Conv (None, 42, 42, 336) 338688 activation_94[0][0]
__________________________________________________________________________________________________
adjust_conv_projection_reduce_6 (None, 42, 42, 336) 338688 activation_93[0][0]
__________________________________________________________________________________________________
reduction_bn_1_reduce_6 (BatchN (None, 42, 42, 336) 1344 reduction_conv_1_reduce_6[0][0]
__________________________________________________________________________________________________
adjust_bn_reduce_6 (BatchNormal (None, 42, 42, 336) 1344 adjust_conv_projection_reduce_6[0
__________________________________________________________________________________________________
activation_95 (Activation) (None, 42, 42, 336) 0 reduction_bn_1_reduce_6[0][0]
__________________________________________________________________________________________________
activation_97 (Activation) (None, 42, 42, 336) 0 adjust_bn_reduce_6[0][0]
__________________________________________________________________________________________________
separable_conv_1_pad_reduction_ (None, 45, 45, 336) 0 activation_95[0][0]
__________________________________________________________________________________________________
separable_conv_1_pad_reduction_ (None, 47, 47, 336) 0 activation_97[0][0]
__________________________________________________________________________________________________
separable_conv_1_reduction_left (None, 21, 21, 336) 121296 separable_conv_1_pad_reduction_le
__________________________________________________________________________________________________
separable_conv_1_reduction_righ (None, 21, 21, 336) 129360 separable_conv_1_pad_reduction_ri
__________________________________________________________________________________________________
separable_conv_1_bn_reduction_l (None, 21, 21, 336) 1344 separable_conv_1_reduction_left1_
__________________________________________________________________________________________________
separable_conv_1_bn_reduction_r (None, 21, 21, 336) 1344 separable_conv_1_reduction_right1
__________________________________________________________________________________________________
activation_96 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_reduction_lef
__________________________________________________________________________________________________
activation_98 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_reduction_rig
__________________________________________________________________________________________________
separable_conv_2_reduction_left (None, 21, 21, 336) 121296 activation_96[0][0]
__________________________________________________________________________________________________
separable_conv_2_reduction_righ (None, 21, 21, 336) 129360 activation_98[0][0]
__________________________________________________________________________________________________
activation_99 (Activation) (None, 42, 42, 336) 0 adjust_bn_reduce_6[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_reduction_l (None, 21, 21, 336) 1344 separable_conv_2_reduction_left1_
__________________________________________________________________________________________________
separable_conv_2_bn_reduction_r (None, 21, 21, 336) 1344 separable_conv_2_reduction_right1
__________________________________________________________________________________________________
separable_conv_1_pad_reduction_ (None, 47, 47, 336) 0 activation_99[0][0]
__________________________________________________________________________________________________
activation_101 (Activation) (None, 42, 42, 336) 0 adjust_bn_reduce_6[0][0]
__________________________________________________________________________________________________
reduction_add_1_reduce_6 (Add) (None, 21, 21, 336) 0 separable_conv_2_bn_reduction_lef
separable_conv_2_bn_reduction_rig
__________________________________________________________________________________________________
separable_conv_1_reduction_righ (None, 21, 21, 336) 129360 separable_conv_1_pad_reduction_ri
__________________________________________________________________________________________________
separable_conv_1_pad_reduction_ (None, 45, 45, 336) 0 activation_101[0][0]
__________________________________________________________________________________________________
activation_103 (Activation) (None, 21, 21, 336) 0 reduction_add_1_reduce_6[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_reduction_r (None, 21, 21, 336) 1344 separable_conv_1_reduction_right2
__________________________________________________________________________________________________
separable_conv_1_reduction_righ (None, 21, 21, 336) 121296 separable_conv_1_pad_reduction_ri
__________________________________________________________________________________________________
separable_conv_1_reduction_left (None, 21, 21, 336) 115920 activation_103[0][0]
__________________________________________________________________________________________________
activation_100 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_reduction_rig
__________________________________________________________________________________________________
separable_conv_1_bn_reduction_r (None, 21, 21, 336) 1344 separable_conv_1_reduction_right3
__________________________________________________________________________________________________
separable_conv_1_bn_reduction_l (None, 21, 21, 336) 1344 separable_conv_1_reduction_left4_
__________________________________________________________________________________________________
reduction_pad_1_reduce_6 (ZeroP (None, 43, 43, 336) 0 reduction_bn_1_reduce_6[0][0]
__________________________________________________________________________________________________
separable_conv_2_reduction_righ (None, 21, 21, 336) 129360 activation_100[0][0]
__________________________________________________________________________________________________
activation_102 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_reduction_rig
__________________________________________________________________________________________________
activation_104 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_reduction_lef
__________________________________________________________________________________________________
reduction_left2_reduce_6 (MaxPo (None, 21, 21, 336) 0 reduction_pad_1_reduce_6[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_reduction_r (None, 21, 21, 336) 1344 separable_conv_2_reduction_right2
__________________________________________________________________________________________________
separable_conv_2_reduction_righ (None, 21, 21, 336) 121296 activation_102[0][0]
__________________________________________________________________________________________________
separable_conv_2_reduction_left (None, 21, 21, 336) 115920 activation_104[0][0]
__________________________________________________________________________________________________
adjust_relu_1_7 (Activation) (None, 42, 42, 1008) 0 normal_concat_4[0][0]
__________________________________________________________________________________________________
reduction_add_2_reduce_6 (Add) (None, 21, 21, 336) 0 reduction_left2_reduce_6[0][0]
separable_conv_2_bn_reduction_rig
__________________________________________________________________________________________________
reduction_left3_reduce_6 (Avera (None, 21, 21, 336) 0 reduction_pad_1_reduce_6[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_reduction_r (None, 21, 21, 336) 1344 separable_conv_2_reduction_right3
__________________________________________________________________________________________________
reduction_left4_reduce_6 (Avera (None, 21, 21, 336) 0 reduction_add_1_reduce_6[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_reduction_l (None, 21, 21, 336) 1344 separable_conv_2_reduction_left4_
__________________________________________________________________________________________________
reduction_right5_reduce_6 (MaxP (None, 21, 21, 336) 0 reduction_pad_1_reduce_6[0][0]
__________________________________________________________________________________________________
zero_padding2d_2 (ZeroPadding2D (None, 43, 43, 1008) 0 adjust_relu_1_7[0][0]
__________________________________________________________________________________________________
reduction_add3_reduce_6 (Add) (None, 21, 21, 336) 0 reduction_left3_reduce_6[0][0]
separable_conv_2_bn_reduction_rig
__________________________________________________________________________________________________
add_2 (Add) (None, 21, 21, 336) 0 reduction_add_2_reduce_6[0][0]
reduction_left4_reduce_6[0][0]
__________________________________________________________________________________________________
reduction_add4_reduce_6 (Add) (None, 21, 21, 336) 0 separable_conv_2_bn_reduction_lef
reduction_right5_reduce_6[0][0]
__________________________________________________________________________________________________
cropping2d_2 (Cropping2D) (None, 42, 42, 1008) 0 zero_padding2d_2[0][0]
__________________________________________________________________________________________________
reduction_concat_reduce_6 (Conc (None, 21, 21, 1344) 0 reduction_add_2_reduce_6[0][0]
reduction_add3_reduce_6[0][0]
add_2[0][0]
reduction_add4_reduce_6[0][0]
__________________________________________________________________________________________________
adjust_avg_pool_1_7 (AveragePoo (None, 21, 21, 1008) 0 adjust_relu_1_7[0][0]
__________________________________________________________________________________________________
adjust_avg_pool_2_7 (AveragePoo (None, 21, 21, 1008) 0 cropping2d_2[0][0]
__________________________________________________________________________________________________
adjust_conv_1_7 (Conv2D) (None, 21, 21, 168) 169344 adjust_avg_pool_1_7[0][0]
__________________________________________________________________________________________________
adjust_conv_2_7 (Conv2D) (None, 21, 21, 168) 169344 adjust_avg_pool_2_7[0][0]
__________________________________________________________________________________________________
activation_105 (Activation) (None, 21, 21, 1344) 0 reduction_concat_reduce_6[0][0]
__________________________________________________________________________________________________
concatenate_2 (Concatenate) (None, 21, 21, 336) 0 adjust_conv_1_7[0][0]
adjust_conv_2_7[0][0]
__________________________________________________________________________________________________
normal_conv_1_7 (Conv2D) (None, 21, 21, 336) 451584 activation_105[0][0]
__________________________________________________________________________________________________
adjust_bn_7 (BatchNormalization (None, 21, 21, 336) 1344 concatenate_2[0][0]
__________________________________________________________________________________________________
normal_bn_1_7 (BatchNormalizati (None, 21, 21, 336) 1344 normal_conv_1_7[0][0]
__________________________________________________________________________________________________
activation_106 (Activation) (None, 21, 21, 336) 0 normal_bn_1_7[0][0]
__________________________________________________________________________________________________
activation_108 (Activation) (None, 21, 21, 336) 0 adjust_bn_7[0][0]
__________________________________________________________________________________________________
activation_110 (Activation) (None, 21, 21, 336) 0 adjust_bn_7[0][0]
__________________________________________________________________________________________________
activation_112 (Activation) (None, 21, 21, 336) 0 adjust_bn_7[0][0]
__________________________________________________________________________________________________
activation_114 (Activation) (None, 21, 21, 336) 0 normal_bn_1_7[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left1_7 (None, 21, 21, 336) 121296 activation_106[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right1_ (None, 21, 21, 336) 115920 activation_108[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left2_7 (None, 21, 21, 336) 121296 activation_110[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right2_ (None, 21, 21, 336) 115920 activation_112[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left5_7 (None, 21, 21, 336) 115920 activation_114[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_1_normal_left1_7[0
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_1_normal_right1_7[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_1_normal_left2_7[0
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_1_normal_right2_7[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_1_normal_left5_7[0
__________________________________________________________________________________________________
activation_107 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_left1_
__________________________________________________________________________________________________
activation_109 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_right1
__________________________________________________________________________________________________
activation_111 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_left2_
__________________________________________________________________________________________________
activation_113 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_right2
__________________________________________________________________________________________________
activation_115 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_left5_
__________________________________________________________________________________________________
separable_conv_2_normal_left1_7 (None, 21, 21, 336) 121296 activation_107[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right1_ (None, 21, 21, 336) 115920 activation_109[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left2_7 (None, 21, 21, 336) 121296 activation_111[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right2_ (None, 21, 21, 336) 115920 activation_113[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left5_7 (None, 21, 21, 336) 115920 activation_115[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_2_normal_left1_7[0
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_2_normal_right1_7[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_2_normal_left2_7[0
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_2_normal_right2_7[
__________________________________________________________________________________________________
normal_left3_7 (AveragePooling2 (None, 21, 21, 336) 0 normal_bn_1_7[0][0]
__________________________________________________________________________________________________
normal_left4_7 (AveragePooling2 (None, 21, 21, 336) 0 adjust_bn_7[0][0]
__________________________________________________________________________________________________
normal_right4_7 (AveragePooling (None, 21, 21, 336) 0 adjust_bn_7[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_2_normal_left5_7[0
__________________________________________________________________________________________________
normal_add_1_7 (Add) (None, 21, 21, 336) 0 separable_conv_2_bn_normal_left1_
separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________
normal_add_2_7 (Add) (None, 21, 21, 336) 0 separable_conv_2_bn_normal_left2_
separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________
normal_add_3_7 (Add) (None, 21, 21, 336) 0 normal_left3_7[0][0]
adjust_bn_7[0][0]
__________________________________________________________________________________________________
normal_add_4_7 (Add) (None, 21, 21, 336) 0 normal_left4_7[0][0]
normal_right4_7[0][0]
__________________________________________________________________________________________________
normal_add_5_7 (Add) (None, 21, 21, 336) 0 separable_conv_2_bn_normal_left5_
normal_bn_1_7[0][0]
__________________________________________________________________________________________________
normal_concat_7 (Concatenate) (None, 21, 21, 2016) 0 adjust_bn_7[0][0]
normal_add_1_7[0][0]
normal_add_2_7[0][0]
normal_add_3_7[0][0]
normal_add_4_7[0][0]
normal_add_5_7[0][0]
__________________________________________________________________________________________________
activation_116 (Activation) (None, 21, 21, 1344) 0 reduction_concat_reduce_6[0][0]
__________________________________________________________________________________________________
activation_117 (Activation) (None, 21, 21, 2016) 0 normal_concat_7[0][0]
__________________________________________________________________________________________________
adjust_conv_projection_8 (Conv2 (None, 21, 21, 336) 451584 activation_116[0][0]
__________________________________________________________________________________________________
normal_conv_1_8 (Conv2D) (None, 21, 21, 336) 677376 activation_117[0][0]
__________________________________________________________________________________________________
adjust_bn_8 (BatchNormalization (None, 21, 21, 336) 1344 adjust_conv_projection_8[0][0]
__________________________________________________________________________________________________
normal_bn_1_8 (BatchNormalizati (None, 21, 21, 336) 1344 normal_conv_1_8[0][0]
__________________________________________________________________________________________________
activation_118 (Activation) (None, 21, 21, 336) 0 normal_bn_1_8[0][0]
__________________________________________________________________________________________________
activation_120 (Activation) (None, 21, 21, 336) 0 adjust_bn_8[0][0]
__________________________________________________________________________________________________
activation_122 (Activation) (None, 21, 21, 336) 0 adjust_bn_8[0][0]
__________________________________________________________________________________________________
activation_124 (Activation) (None, 21, 21, 336) 0 adjust_bn_8[0][0]
__________________________________________________________________________________________________
activation_126 (Activation) (None, 21, 21, 336) 0 normal_bn_1_8[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left1_8 (None, 21, 21, 336) 121296 activation_118[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right1_ (None, 21, 21, 336) 115920 activation_120[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left2_8 (None, 21, 21, 336) 121296 activation_122[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right2_ (None, 21, 21, 336) 115920 activation_124[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left5_8 (None, 21, 21, 336) 115920 activation_126[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_1_normal_left1_8[0
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_1_normal_right1_8[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_1_normal_left2_8[0
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_1_normal_right2_8[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_1_normal_left5_8[0
__________________________________________________________________________________________________
activation_119 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_left1_
__________________________________________________________________________________________________
activation_121 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_right1
__________________________________________________________________________________________________
activation_123 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_left2_
__________________________________________________________________________________________________
activation_125 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_right2
__________________________________________________________________________________________________
activation_127 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_left5_
__________________________________________________________________________________________________
separable_conv_2_normal_left1_8 (None, 21, 21, 336) 121296 activation_119[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right1_ (None, 21, 21, 336) 115920 activation_121[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left2_8 (None, 21, 21, 336) 121296 activation_123[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right2_ (None, 21, 21, 336) 115920 activation_125[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left5_8 (None, 21, 21, 336) 115920 activation_127[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_2_normal_left1_8[0
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_2_normal_right1_8[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_2_normal_left2_8[0
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_2_normal_right2_8[
__________________________________________________________________________________________________
normal_left3_8 (AveragePooling2 (None, 21, 21, 336) 0 normal_bn_1_8[0][0]
__________________________________________________________________________________________________
normal_left4_8 (AveragePooling2 (None, 21, 21, 336) 0 adjust_bn_8[0][0]
__________________________________________________________________________________________________
normal_right4_8 (AveragePooling (None, 21, 21, 336) 0 adjust_bn_8[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_2_normal_left5_8[0
__________________________________________________________________________________________________
normal_add_1_8 (Add) (None, 21, 21, 336) 0 separable_conv_2_bn_normal_left1_
separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________
normal_add_2_8 (Add) (None, 21, 21, 336) 0 separable_conv_2_bn_normal_left2_
separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________
normal_add_3_8 (Add) (None, 21, 21, 336) 0 normal_left3_8[0][0]
adjust_bn_8[0][0]
__________________________________________________________________________________________________
normal_add_4_8 (Add) (None, 21, 21, 336) 0 normal_left4_8[0][0]
normal_right4_8[0][0]
__________________________________________________________________________________________________
normal_add_5_8 (Add) (None, 21, 21, 336) 0 separable_conv_2_bn_normal_left5_
normal_bn_1_8[0][0]
__________________________________________________________________________________________________
normal_concat_8 (Concatenate) (None, 21, 21, 2016) 0 adjust_bn_8[0][0]
normal_add_1_8[0][0]
normal_add_2_8[0][0]
normal_add_3_8[0][0]
normal_add_4_8[0][0]
normal_add_5_8[0][0]
__________________________________________________________________________________________________
activation_128 (Activation) (None, 21, 21, 2016) 0 normal_concat_7[0][0]
__________________________________________________________________________________________________
activation_129 (Activation) (None, 21, 21, 2016) 0 normal_concat_8[0][0]
__________________________________________________________________________________________________
adjust_conv_projection_9 (Conv2 (None, 21, 21, 336) 677376 activation_128[0][0]
__________________________________________________________________________________________________
normal_conv_1_9 (Conv2D) (None, 21, 21, 336) 677376 activation_129[0][0]
__________________________________________________________________________________________________
adjust_bn_9 (BatchNormalization (None, 21, 21, 336) 1344 adjust_conv_projection_9[0][0]
__________________________________________________________________________________________________
normal_bn_1_9 (BatchNormalizati (None, 21, 21, 336) 1344 normal_conv_1_9[0][0]
__________________________________________________________________________________________________
activation_130 (Activation) (None, 21, 21, 336) 0 normal_bn_1_9[0][0]
__________________________________________________________________________________________________
activation_132 (Activation) (None, 21, 21, 336) 0 adjust_bn_9[0][0]
__________________________________________________________________________________________________
activation_134 (Activation) (None, 21, 21, 336) 0 adjust_bn_9[0][0]
__________________________________________________________________________________________________
activation_136 (Activation) (None, 21, 21, 336) 0 adjust_bn_9[0][0]
__________________________________________________________________________________________________
activation_138 (Activation) (None, 21, 21, 336) 0 normal_bn_1_9[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left1_9 (None, 21, 21, 336) 121296 activation_130[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right1_ (None, 21, 21, 336) 115920 activation_132[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left2_9 (None, 21, 21, 336) 121296 activation_134[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right2_ (None, 21, 21, 336) 115920 activation_136[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left5_9 (None, 21, 21, 336) 115920 activation_138[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_1_normal_left1_9[0
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_1_normal_right1_9[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_1_normal_left2_9[0
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_1_normal_right2_9[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_1_normal_left5_9[0
__________________________________________________________________________________________________
activation_131 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_left1_
__________________________________________________________________________________________________
activation_133 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_right1
__________________________________________________________________________________________________
activation_135 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_left2_
__________________________________________________________________________________________________
activation_137 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_right2
__________________________________________________________________________________________________
activation_139 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_left5_
__________________________________________________________________________________________________
separable_conv_2_normal_left1_9 (None, 21, 21, 336) 121296 activation_131[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right1_ (None, 21, 21, 336) 115920 activation_133[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left2_9 (None, 21, 21, 336) 121296 activation_135[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right2_ (None, 21, 21, 336) 115920 activation_137[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left5_9 (None, 21, 21, 336) 115920 activation_139[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_2_normal_left1_9[0
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_2_normal_right1_9[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_2_normal_left2_9[0
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_2_normal_right2_9[
__________________________________________________________________________________________________
normal_left3_9 (AveragePooling2 (None, 21, 21, 336) 0 normal_bn_1_9[0][0]
__________________________________________________________________________________________________
normal_left4_9 (AveragePooling2 (None, 21, 21, 336) 0 adjust_bn_9[0][0]
__________________________________________________________________________________________________
normal_right4_9 (AveragePooling (None, 21, 21, 336) 0 adjust_bn_9[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_2_normal_left5_9[0
__________________________________________________________________________________________________
normal_add_1_9 (Add) (None, 21, 21, 336) 0 separable_conv_2_bn_normal_left1_
separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________
normal_add_2_9 (Add) (None, 21, 21, 336) 0 separable_conv_2_bn_normal_left2_
separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________
normal_add_3_9 (Add) (None, 21, 21, 336) 0 normal_left3_9[0][0]
adjust_bn_9[0][0]
__________________________________________________________________________________________________
normal_add_4_9 (Add) (None, 21, 21, 336) 0 normal_left4_9[0][0]
normal_right4_9[0][0]
__________________________________________________________________________________________________
normal_add_5_9 (Add) (None, 21, 21, 336) 0 separable_conv_2_bn_normal_left5_
normal_bn_1_9[0][0]
__________________________________________________________________________________________________
normal_concat_9 (Concatenate) (None, 21, 21, 2016) 0 adjust_bn_9[0][0]
normal_add_1_9[0][0]
normal_add_2_9[0][0]
normal_add_3_9[0][0]
normal_add_4_9[0][0]
normal_add_5_9[0][0]
__________________________________________________________________________________________________
activation_140 (Activation) (None, 21, 21, 2016) 0 normal_concat_8[0][0]
__________________________________________________________________________________________________
activation_141 (Activation) (None, 21, 21, 2016) 0 normal_concat_9[0][0]
__________________________________________________________________________________________________
adjust_conv_projection_10 (Conv (None, 21, 21, 336) 677376 activation_140[0][0]
__________________________________________________________________________________________________
normal_conv_1_10 (Conv2D) (None, 21, 21, 336) 677376 activation_141[0][0]
__________________________________________________________________________________________________
adjust_bn_10 (BatchNormalizatio (None, 21, 21, 336) 1344 adjust_conv_projection_10[0][0]
__________________________________________________________________________________________________
normal_bn_1_10 (BatchNormalizat (None, 21, 21, 336) 1344 normal_conv_1_10[0][0]
__________________________________________________________________________________________________
activation_142 (Activation) (None, 21, 21, 336) 0 normal_bn_1_10[0][0]
__________________________________________________________________________________________________
activation_144 (Activation) (None, 21, 21, 336) 0 adjust_bn_10[0][0]
__________________________________________________________________________________________________
activation_146 (Activation) (None, 21, 21, 336) 0 adjust_bn_10[0][0]
__________________________________________________________________________________________________
activation_148 (Activation) (None, 21, 21, 336) 0 adjust_bn_10[0][0]
__________________________________________________________________________________________________
activation_150 (Activation) (None, 21, 21, 336) 0 normal_bn_1_10[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left1_1 (None, 21, 21, 336) 121296 activation_142[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right1_ (None, 21, 21, 336) 115920 activation_144[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left2_1 (None, 21, 21, 336) 121296 activation_146[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right2_ (None, 21, 21, 336) 115920 activation_148[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left5_1 (None, 21, 21, 336) 115920 activation_150[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_1_normal_left1_10[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_1_normal_right1_10
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_1_normal_left2_10[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_1_normal_right2_10
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_1_normal_left5_10[
__________________________________________________________________________________________________
activation_143 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_left1_
__________________________________________________________________________________________________
activation_145 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_right1
__________________________________________________________________________________________________
activation_147 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_left2_
__________________________________________________________________________________________________
activation_149 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_right2
__________________________________________________________________________________________________
activation_151 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_left5_
__________________________________________________________________________________________________
separable_conv_2_normal_left1_1 (None, 21, 21, 336) 121296 activation_143[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right1_ (None, 21, 21, 336) 115920 activation_145[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left2_1 (None, 21, 21, 336) 121296 activation_147[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right2_ (None, 21, 21, 336) 115920 activation_149[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left5_1 (None, 21, 21, 336) 115920 activation_151[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_2_normal_left1_10[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_2_normal_right1_10
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_2_normal_left2_10[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_2_normal_right2_10
__________________________________________________________________________________________________
normal_left3_10 (AveragePooling (None, 21, 21, 336) 0 normal_bn_1_10[0][0]
__________________________________________________________________________________________________
normal_left4_10 (AveragePooling (None, 21, 21, 336) 0 adjust_bn_10[0][0]
__________________________________________________________________________________________________
normal_right4_10 (AveragePoolin (None, 21, 21, 336) 0 adjust_bn_10[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_2_normal_left5_10[
__________________________________________________________________________________________________
normal_add_1_10 (Add) (None, 21, 21, 336) 0 separable_conv_2_bn_normal_left1_
separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________
normal_add_2_10 (Add) (None, 21, 21, 336) 0 separable_conv_2_bn_normal_left2_
separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________
normal_add_3_10 (Add) (None, 21, 21, 336) 0 normal_left3_10[0][0]
adjust_bn_10[0][0]
__________________________________________________________________________________________________
normal_add_4_10 (Add) (None, 21, 21, 336) 0 normal_left4_10[0][0]
normal_right4_10[0][0]
__________________________________________________________________________________________________
normal_add_5_10 (Add) (None, 21, 21, 336) 0 separable_conv_2_bn_normal_left5_
normal_bn_1_10[0][0]
__________________________________________________________________________________________________
normal_concat_10 (Concatenate) (None, 21, 21, 2016) 0 adjust_bn_10[0][0]
normal_add_1_10[0][0]
normal_add_2_10[0][0]
normal_add_3_10[0][0]
normal_add_4_10[0][0]
normal_add_5_10[0][0]
__________________________________________________________________________________________________
activation_152 (Activation) (None, 21, 21, 2016) 0 normal_concat_9[0][0]
__________________________________________________________________________________________________
activation_153 (Activation) (None, 21, 21, 2016) 0 normal_concat_10[0][0]
__________________________________________________________________________________________________
adjust_conv_projection_11 (Conv (None, 21, 21, 336) 677376 activation_152[0][0]
__________________________________________________________________________________________________
normal_conv_1_11 (Conv2D) (None, 21, 21, 336) 677376 activation_153[0][0]
__________________________________________________________________________________________________
adjust_bn_11 (BatchNormalizatio (None, 21, 21, 336) 1344 adjust_conv_projection_11[0][0]
__________________________________________________________________________________________________
normal_bn_1_11 (BatchNormalizat (None, 21, 21, 336) 1344 normal_conv_1_11[0][0]
__________________________________________________________________________________________________
activation_154 (Activation) (None, 21, 21, 336) 0 normal_bn_1_11[0][0]
__________________________________________________________________________________________________
activation_156 (Activation) (None, 21, 21, 336) 0 adjust_bn_11[0][0]
__________________________________________________________________________________________________
activation_158 (Activation) (None, 21, 21, 336) 0 adjust_bn_11[0][0]
__________________________________________________________________________________________________
activation_160 (Activation) (None, 21, 21, 336) 0 adjust_bn_11[0][0]
__________________________________________________________________________________________________
activation_162 (Activation) (None, 21, 21, 336) 0 normal_bn_1_11[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left1_1 (None, 21, 21, 336) 121296 activation_154[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right1_ (None, 21, 21, 336) 115920 activation_156[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left2_1 (None, 21, 21, 336) 121296 activation_158[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right2_ (None, 21, 21, 336) 115920 activation_160[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left5_1 (None, 21, 21, 336) 115920 activation_162[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_1_normal_left1_11[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_1_normal_right1_11
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_1_normal_left2_11[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_1_normal_right2_11
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_1_normal_left5_11[
__________________________________________________________________________________________________
activation_155 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_left1_
__________________________________________________________________________________________________
activation_157 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_right1
__________________________________________________________________________________________________
activation_159 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_left2_
__________________________________________________________________________________________________
activation_161 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_right2
__________________________________________________________________________________________________
activation_163 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_left5_
__________________________________________________________________________________________________
separable_conv_2_normal_left1_1 (None, 21, 21, 336) 121296 activation_155[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right1_ (None, 21, 21, 336) 115920 activation_157[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left2_1 (None, 21, 21, 336) 121296 activation_159[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right2_ (None, 21, 21, 336) 115920 activation_161[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left5_1 (None, 21, 21, 336) 115920 activation_163[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_2_normal_left1_11[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_2_normal_right1_11
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_2_normal_left2_11[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_2_normal_right2_11
__________________________________________________________________________________________________
normal_left3_11 (AveragePooling (None, 21, 21, 336) 0 normal_bn_1_11[0][0]
__________________________________________________________________________________________________
normal_left4_11 (AveragePooling (None, 21, 21, 336) 0 adjust_bn_11[0][0]
__________________________________________________________________________________________________
normal_right4_11 (AveragePoolin (None, 21, 21, 336) 0 adjust_bn_11[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_2_normal_left5_11[
__________________________________________________________________________________________________
normal_add_1_11 (Add) (None, 21, 21, 336) 0 separable_conv_2_bn_normal_left1_
separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________
normal_add_2_11 (Add) (None, 21, 21, 336) 0 separable_conv_2_bn_normal_left2_
separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________
normal_add_3_11 (Add) (None, 21, 21, 336) 0 normal_left3_11[0][0]
adjust_bn_11[0][0]
__________________________________________________________________________________________________
normal_add_4_11 (Add) (None, 21, 21, 336) 0 normal_left4_11[0][0]
normal_right4_11[0][0]
__________________________________________________________________________________________________
normal_add_5_11 (Add) (None, 21, 21, 336) 0 separable_conv_2_bn_normal_left5_
normal_bn_1_11[0][0]
__________________________________________________________________________________________________
normal_concat_11 (Concatenate) (None, 21, 21, 2016) 0 adjust_bn_11[0][0]
normal_add_1_11[0][0]
normal_add_2_11[0][0]
normal_add_3_11[0][0]
normal_add_4_11[0][0]
normal_add_5_11[0][0]
__________________________________________________________________________________________________
activation_164 (Activation) (None, 21, 21, 2016) 0 normal_concat_10[0][0]
__________________________________________________________________________________________________
activation_165 (Activation) (None, 21, 21, 2016) 0 normal_concat_11[0][0]
__________________________________________________________________________________________________
adjust_conv_projection_12 (Conv (None, 21, 21, 336) 677376 activation_164[0][0]
__________________________________________________________________________________________________
normal_conv_1_12 (Conv2D) (None, 21, 21, 336) 677376 activation_165[0][0]
__________________________________________________________________________________________________
adjust_bn_12 (BatchNormalizatio (None, 21, 21, 336) 1344 adjust_conv_projection_12[0][0]
__________________________________________________________________________________________________
normal_bn_1_12 (BatchNormalizat (None, 21, 21, 336) 1344 normal_conv_1_12[0][0]
__________________________________________________________________________________________________
activation_166 (Activation) (None, 21, 21, 336) 0 normal_bn_1_12[0][0]
__________________________________________________________________________________________________
activation_168 (Activation) (None, 21, 21, 336) 0 adjust_bn_12[0][0]
__________________________________________________________________________________________________
activation_170 (Activation) (None, 21, 21, 336) 0 adjust_bn_12[0][0]
__________________________________________________________________________________________________
activation_172 (Activation) (None, 21, 21, 336) 0 adjust_bn_12[0][0]
__________________________________________________________________________________________________
activation_174 (Activation) (None, 21, 21, 336) 0 normal_bn_1_12[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left1_1 (None, 21, 21, 336) 121296 activation_166[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right1_ (None, 21, 21, 336) 115920 activation_168[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left2_1 (None, 21, 21, 336) 121296 activation_170[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right2_ (None, 21, 21, 336) 115920 activation_172[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left5_1 (None, 21, 21, 336) 115920 activation_174[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_1_normal_left1_12[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_1_normal_right1_12
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_1_normal_left2_12[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_1_normal_right2_12
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_1_normal_left5_12[
__________________________________________________________________________________________________
activation_167 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_left1_
__________________________________________________________________________________________________
activation_169 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_right1
__________________________________________________________________________________________________
activation_171 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_left2_
__________________________________________________________________________________________________
activation_173 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_right2
__________________________________________________________________________________________________
activation_175 (Activation) (None, 21, 21, 336) 0 separable_conv_1_bn_normal_left5_
__________________________________________________________________________________________________
separable_conv_2_normal_left1_1 (None, 21, 21, 336) 121296 activation_167[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right1_ (None, 21, 21, 336) 115920 activation_169[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left2_1 (None, 21, 21, 336) 121296 activation_171[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right2_ (None, 21, 21, 336) 115920 activation_173[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left5_1 (None, 21, 21, 336) 115920 activation_175[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_2_normal_left1_12[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_2_normal_right1_12
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_2_normal_left2_12[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 21, 21, 336) 1344 separable_conv_2_normal_right2_12
__________________________________________________________________________________________________
normal_left3_12 (AveragePooling (None, 21, 21, 336) 0 normal_bn_1_12[0][0]
__________________________________________________________________________________________________
normal_left4_12 (AveragePooling (None, 21, 21, 336) 0 adjust_bn_12[0][0]
__________________________________________________________________________________________________
normal_right4_12 (AveragePoolin (None, 21, 21, 336) 0 adjust_bn_12[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 21, 21, 336) 1344 separable_conv_2_normal_left5_12[
__________________________________________________________________________________________________
normal_add_1_12 (Add) (None, 21, 21, 336) 0 separable_conv_2_bn_normal_left1_
separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________
normal_add_2_12 (Add) (None, 21, 21, 336) 0 separable_conv_2_bn_normal_left2_
separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________
normal_add_3_12 (Add) (None, 21, 21, 336) 0 normal_left3_12[0][0]
adjust_bn_12[0][0]
__________________________________________________________________________________________________
normal_add_4_12 (Add) (None, 21, 21, 336) 0 normal_left4_12[0][0]
normal_right4_12[0][0]
__________________________________________________________________________________________________
normal_add_5_12 (Add) (None, 21, 21, 336) 0 separable_conv_2_bn_normal_left5_
normal_bn_1_12[0][0]
__________________________________________________________________________________________________
normal_concat_12 (Concatenate) (None, 21, 21, 2016) 0 adjust_bn_12[0][0]
normal_add_1_12[0][0]
normal_add_2_12[0][0]
normal_add_3_12[0][0]
normal_add_4_12[0][0]
normal_add_5_12[0][0]
__________________________________________________________________________________________________
activation_177 (Activation) (None, 21, 21, 2016) 0 normal_concat_12[0][0]
__________________________________________________________________________________________________
activation_176 (Activation) (None, 21, 21, 2016) 0 normal_concat_11[0][0]
__________________________________________________________________________________________________
reduction_conv_1_reduce_12 (Con (None, 21, 21, 672) 1354752 activation_177[0][0]
__________________________________________________________________________________________________
adjust_conv_projection_reduce_1 (None, 21, 21, 672) 1354752 activation_176[0][0]
__________________________________________________________________________________________________
reduction_bn_1_reduce_12 (Batch (None, 21, 21, 672) 2688 reduction_conv_1_reduce_12[0][0]
__________________________________________________________________________________________________
adjust_bn_reduce_12 (BatchNorma (None, 21, 21, 672) 2688 adjust_conv_projection_reduce_12[
__________________________________________________________________________________________________
activation_178 (Activation) (None, 21, 21, 672) 0 reduction_bn_1_reduce_12[0][0]
__________________________________________________________________________________________________
activation_180 (Activation) (None, 21, 21, 672) 0 adjust_bn_reduce_12[0][0]
__________________________________________________________________________________________________
separable_conv_1_pad_reduction_ (None, 25, 25, 672) 0 activation_178[0][0]
__________________________________________________________________________________________________
separable_conv_1_pad_reduction_ (None, 27, 27, 672) 0 activation_180[0][0]
__________________________________________________________________________________________________
separable_conv_1_reduction_left (None, 11, 11, 672) 468384 separable_conv_1_pad_reduction_le
__________________________________________________________________________________________________
separable_conv_1_reduction_righ (None, 11, 11, 672) 484512 separable_conv_1_pad_reduction_ri
__________________________________________________________________________________________________
separable_conv_1_bn_reduction_l (None, 11, 11, 672) 2688 separable_conv_1_reduction_left1_
__________________________________________________________________________________________________
separable_conv_1_bn_reduction_r (None, 11, 11, 672) 2688 separable_conv_1_reduction_right1
__________________________________________________________________________________________________
activation_179 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_reduction_lef
__________________________________________________________________________________________________
activation_181 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_reduction_rig
__________________________________________________________________________________________________
separable_conv_2_reduction_left (None, 11, 11, 672) 468384 activation_179[0][0]
__________________________________________________________________________________________________
separable_conv_2_reduction_righ (None, 11, 11, 672) 484512 activation_181[0][0]
__________________________________________________________________________________________________
activation_182 (Activation) (None, 21, 21, 672) 0 adjust_bn_reduce_12[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_reduction_l (None, 11, 11, 672) 2688 separable_conv_2_reduction_left1_
__________________________________________________________________________________________________
separable_conv_2_bn_reduction_r (None, 11, 11, 672) 2688 separable_conv_2_reduction_right1
__________________________________________________________________________________________________
separable_conv_1_pad_reduction_ (None, 27, 27, 672) 0 activation_182[0][0]
__________________________________________________________________________________________________
activation_184 (Activation) (None, 21, 21, 672) 0 adjust_bn_reduce_12[0][0]
__________________________________________________________________________________________________
reduction_add_1_reduce_12 (Add) (None, 11, 11, 672) 0 separable_conv_2_bn_reduction_lef
separable_conv_2_bn_reduction_rig
__________________________________________________________________________________________________
separable_conv_1_reduction_righ (None, 11, 11, 672) 484512 separable_conv_1_pad_reduction_ri
__________________________________________________________________________________________________
separable_conv_1_pad_reduction_ (None, 25, 25, 672) 0 activation_184[0][0]
__________________________________________________________________________________________________
activation_186 (Activation) (None, 11, 11, 672) 0 reduction_add_1_reduce_12[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_reduction_r (None, 11, 11, 672) 2688 separable_conv_1_reduction_right2
__________________________________________________________________________________________________
separable_conv_1_reduction_righ (None, 11, 11, 672) 468384 separable_conv_1_pad_reduction_ri
__________________________________________________________________________________________________
separable_conv_1_reduction_left (None, 11, 11, 672) 457632 activation_186[0][0]
__________________________________________________________________________________________________
activation_183 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_reduction_rig
__________________________________________________________________________________________________
separable_conv_1_bn_reduction_r (None, 11, 11, 672) 2688 separable_conv_1_reduction_right3
__________________________________________________________________________________________________
separable_conv_1_bn_reduction_l (None, 11, 11, 672) 2688 separable_conv_1_reduction_left4_
__________________________________________________________________________________________________
reduction_pad_1_reduce_12 (Zero (None, 23, 23, 672) 0 reduction_bn_1_reduce_12[0][0]
__________________________________________________________________________________________________
separable_conv_2_reduction_righ (None, 11, 11, 672) 484512 activation_183[0][0]
__________________________________________________________________________________________________
activation_185 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_reduction_rig
__________________________________________________________________________________________________
activation_187 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_reduction_lef
__________________________________________________________________________________________________
reduction_left2_reduce_12 (MaxP (None, 11, 11, 672) 0 reduction_pad_1_reduce_12[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_reduction_r (None, 11, 11, 672) 2688 separable_conv_2_reduction_right2
__________________________________________________________________________________________________
separable_conv_2_reduction_righ (None, 11, 11, 672) 468384 activation_185[0][0]
__________________________________________________________________________________________________
separable_conv_2_reduction_left (None, 11, 11, 672) 457632 activation_187[0][0]
__________________________________________________________________________________________________
adjust_relu_1_13 (Activation) (None, 21, 21, 2016) 0 normal_concat_11[0][0]
__________________________________________________________________________________________________
reduction_add_2_reduce_12 (Add) (None, 11, 11, 672) 0 reduction_left2_reduce_12[0][0]
separable_conv_2_bn_reduction_rig
__________________________________________________________________________________________________
reduction_left3_reduce_12 (Aver (None, 11, 11, 672) 0 reduction_pad_1_reduce_12[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_reduction_r (None, 11, 11, 672) 2688 separable_conv_2_reduction_right3
__________________________________________________________________________________________________
reduction_left4_reduce_12 (Aver (None, 11, 11, 672) 0 reduction_add_1_reduce_12[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_reduction_l (None, 11, 11, 672) 2688 separable_conv_2_reduction_left4_
__________________________________________________________________________________________________
reduction_right5_reduce_12 (Max (None, 11, 11, 672) 0 reduction_pad_1_reduce_12[0][0]
__________________________________________________________________________________________________
zero_padding2d_3 (ZeroPadding2D (None, 22, 22, 2016) 0 adjust_relu_1_13[0][0]
__________________________________________________________________________________________________
reduction_add3_reduce_12 (Add) (None, 11, 11, 672) 0 reduction_left3_reduce_12[0][0]
separable_conv_2_bn_reduction_rig
__________________________________________________________________________________________________
add_3 (Add) (None, 11, 11, 672) 0 reduction_add_2_reduce_12[0][0]
reduction_left4_reduce_12[0][0]
__________________________________________________________________________________________________
reduction_add4_reduce_12 (Add) (None, 11, 11, 672) 0 separable_conv_2_bn_reduction_lef
reduction_right5_reduce_12[0][0]
__________________________________________________________________________________________________
cropping2d_3 (Cropping2D) (None, 21, 21, 2016) 0 zero_padding2d_3[0][0]
__________________________________________________________________________________________________
reduction_concat_reduce_12 (Con (None, 11, 11, 2688) 0 reduction_add_2_reduce_12[0][0]
reduction_add3_reduce_12[0][0]
add_3[0][0]
reduction_add4_reduce_12[0][0]
__________________________________________________________________________________________________
adjust_avg_pool_1_13 (AveragePo (None, 11, 11, 2016) 0 adjust_relu_1_13[0][0]
__________________________________________________________________________________________________
adjust_avg_pool_2_13 (AveragePo (None, 11, 11, 2016) 0 cropping2d_3[0][0]
__________________________________________________________________________________________________
adjust_conv_1_13 (Conv2D) (None, 11, 11, 336) 677376 adjust_avg_pool_1_13[0][0]
__________________________________________________________________________________________________
adjust_conv_2_13 (Conv2D) (None, 11, 11, 336) 677376 adjust_avg_pool_2_13[0][0]
__________________________________________________________________________________________________
activation_188 (Activation) (None, 11, 11, 2688) 0 reduction_concat_reduce_12[0][0]
__________________________________________________________________________________________________
concatenate_3 (Concatenate) (None, 11, 11, 672) 0 adjust_conv_1_13[0][0]
adjust_conv_2_13[0][0]
__________________________________________________________________________________________________
normal_conv_1_13 (Conv2D) (None, 11, 11, 672) 1806336 activation_188[0][0]
__________________________________________________________________________________________________
adjust_bn_13 (BatchNormalizatio (None, 11, 11, 672) 2688 concatenate_3[0][0]
__________________________________________________________________________________________________
normal_bn_1_13 (BatchNormalizat (None, 11, 11, 672) 2688 normal_conv_1_13[0][0]
__________________________________________________________________________________________________
activation_189 (Activation) (None, 11, 11, 672) 0 normal_bn_1_13[0][0]
__________________________________________________________________________________________________
activation_191 (Activation) (None, 11, 11, 672) 0 adjust_bn_13[0][0]
__________________________________________________________________________________________________
activation_193 (Activation) (None, 11, 11, 672) 0 adjust_bn_13[0][0]
__________________________________________________________________________________________________
activation_195 (Activation) (None, 11, 11, 672) 0 adjust_bn_13[0][0]
__________________________________________________________________________________________________
activation_197 (Activation) (None, 11, 11, 672) 0 normal_bn_1_13[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left1_1 (None, 11, 11, 672) 468384 activation_189[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right1_ (None, 11, 11, 672) 457632 activation_191[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left2_1 (None, 11, 11, 672) 468384 activation_193[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right2_ (None, 11, 11, 672) 457632 activation_195[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left5_1 (None, 11, 11, 672) 457632 activation_197[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_1_normal_left1_13[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_1_normal_right1_13
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_1_normal_left2_13[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_1_normal_right2_13
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_1_normal_left5_13[
__________________________________________________________________________________________________
activation_190 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_left1_
__________________________________________________________________________________________________
activation_192 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_right1
__________________________________________________________________________________________________
activation_194 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_left2_
__________________________________________________________________________________________________
activation_196 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_right2
__________________________________________________________________________________________________
activation_198 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_left5_
__________________________________________________________________________________________________
separable_conv_2_normal_left1_1 (None, 11, 11, 672) 468384 activation_190[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right1_ (None, 11, 11, 672) 457632 activation_192[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left2_1 (None, 11, 11, 672) 468384 activation_194[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right2_ (None, 11, 11, 672) 457632 activation_196[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left5_1 (None, 11, 11, 672) 457632 activation_198[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_2_normal_left1_13[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_2_normal_right1_13
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_2_normal_left2_13[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_2_normal_right2_13
__________________________________________________________________________________________________
normal_left3_13 (AveragePooling (None, 11, 11, 672) 0 normal_bn_1_13[0][0]
__________________________________________________________________________________________________
normal_left4_13 (AveragePooling (None, 11, 11, 672) 0 adjust_bn_13[0][0]
__________________________________________________________________________________________________
normal_right4_13 (AveragePoolin (None, 11, 11, 672) 0 adjust_bn_13[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_2_normal_left5_13[
__________________________________________________________________________________________________
normal_add_1_13 (Add) (None, 11, 11, 672) 0 separable_conv_2_bn_normal_left1_
separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________
normal_add_2_13 (Add) (None, 11, 11, 672) 0 separable_conv_2_bn_normal_left2_
separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________
normal_add_3_13 (Add) (None, 11, 11, 672) 0 normal_left3_13[0][0]
adjust_bn_13[0][0]
__________________________________________________________________________________________________
normal_add_4_13 (Add) (None, 11, 11, 672) 0 normal_left4_13[0][0]
normal_right4_13[0][0]
__________________________________________________________________________________________________
normal_add_5_13 (Add) (None, 11, 11, 672) 0 separable_conv_2_bn_normal_left5_
normal_bn_1_13[0][0]
__________________________________________________________________________________________________
normal_concat_13 (Concatenate) (None, 11, 11, 4032) 0 adjust_bn_13[0][0]
normal_add_1_13[0][0]
normal_add_2_13[0][0]
normal_add_3_13[0][0]
normal_add_4_13[0][0]
normal_add_5_13[0][0]
__________________________________________________________________________________________________
activation_199 (Activation) (None, 11, 11, 2688) 0 reduction_concat_reduce_12[0][0]
__________________________________________________________________________________________________
activation_200 (Activation) (None, 11, 11, 4032) 0 normal_concat_13[0][0]
__________________________________________________________________________________________________
adjust_conv_projection_14 (Conv (None, 11, 11, 672) 1806336 activation_199[0][0]
__________________________________________________________________________________________________
normal_conv_1_14 (Conv2D) (None, 11, 11, 672) 2709504 activation_200[0][0]
__________________________________________________________________________________________________
adjust_bn_14 (BatchNormalizatio (None, 11, 11, 672) 2688 adjust_conv_projection_14[0][0]
__________________________________________________________________________________________________
normal_bn_1_14 (BatchNormalizat (None, 11, 11, 672) 2688 normal_conv_1_14[0][0]
__________________________________________________________________________________________________
activation_201 (Activation) (None, 11, 11, 672) 0 normal_bn_1_14[0][0]
__________________________________________________________________________________________________
activation_203 (Activation) (None, 11, 11, 672) 0 adjust_bn_14[0][0]
__________________________________________________________________________________________________
activation_205 (Activation) (None, 11, 11, 672) 0 adjust_bn_14[0][0]
__________________________________________________________________________________________________
activation_207 (Activation) (None, 11, 11, 672) 0 adjust_bn_14[0][0]
__________________________________________________________________________________________________
activation_209 (Activation) (None, 11, 11, 672) 0 normal_bn_1_14[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left1_1 (None, 11, 11, 672) 468384 activation_201[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right1_ (None, 11, 11, 672) 457632 activation_203[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left2_1 (None, 11, 11, 672) 468384 activation_205[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right2_ (None, 11, 11, 672) 457632 activation_207[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left5_1 (None, 11, 11, 672) 457632 activation_209[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_1_normal_left1_14[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_1_normal_right1_14
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_1_normal_left2_14[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_1_normal_right2_14
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_1_normal_left5_14[
__________________________________________________________________________________________________
activation_202 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_left1_
__________________________________________________________________________________________________
activation_204 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_right1
__________________________________________________________________________________________________
activation_206 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_left2_
__________________________________________________________________________________________________
activation_208 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_right2
__________________________________________________________________________________________________
activation_210 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_left5_
__________________________________________________________________________________________________
separable_conv_2_normal_left1_1 (None, 11, 11, 672) 468384 activation_202[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right1_ (None, 11, 11, 672) 457632 activation_204[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left2_1 (None, 11, 11, 672) 468384 activation_206[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right2_ (None, 11, 11, 672) 457632 activation_208[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left5_1 (None, 11, 11, 672) 457632 activation_210[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_2_normal_left1_14[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_2_normal_right1_14
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_2_normal_left2_14[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_2_normal_right2_14
__________________________________________________________________________________________________
normal_left3_14 (AveragePooling (None, 11, 11, 672) 0 normal_bn_1_14[0][0]
__________________________________________________________________________________________________
normal_left4_14 (AveragePooling (None, 11, 11, 672) 0 adjust_bn_14[0][0]
__________________________________________________________________________________________________
normal_right4_14 (AveragePoolin (None, 11, 11, 672) 0 adjust_bn_14[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_2_normal_left5_14[
__________________________________________________________________________________________________
normal_add_1_14 (Add) (None, 11, 11, 672) 0 separable_conv_2_bn_normal_left1_
separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________
normal_add_2_14 (Add) (None, 11, 11, 672) 0 separable_conv_2_bn_normal_left2_
separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________
normal_add_3_14 (Add) (None, 11, 11, 672) 0 normal_left3_14[0][0]
adjust_bn_14[0][0]
__________________________________________________________________________________________________
normal_add_4_14 (Add) (None, 11, 11, 672) 0 normal_left4_14[0][0]
normal_right4_14[0][0]
__________________________________________________________________________________________________
normal_add_5_14 (Add) (None, 11, 11, 672) 0 separable_conv_2_bn_normal_left5_
normal_bn_1_14[0][0]
__________________________________________________________________________________________________
normal_concat_14 (Concatenate) (None, 11, 11, 4032) 0 adjust_bn_14[0][0]
normal_add_1_14[0][0]
normal_add_2_14[0][0]
normal_add_3_14[0][0]
normal_add_4_14[0][0]
normal_add_5_14[0][0]
__________________________________________________________________________________________________
activation_211 (Activation) (None, 11, 11, 4032) 0 normal_concat_13[0][0]
__________________________________________________________________________________________________
activation_212 (Activation) (None, 11, 11, 4032) 0 normal_concat_14[0][0]
__________________________________________________________________________________________________
adjust_conv_projection_15 (Conv (None, 11, 11, 672) 2709504 activation_211[0][0]
__________________________________________________________________________________________________
normal_conv_1_15 (Conv2D) (None, 11, 11, 672) 2709504 activation_212[0][0]
__________________________________________________________________________________________________
adjust_bn_15 (BatchNormalizatio (None, 11, 11, 672) 2688 adjust_conv_projection_15[0][0]
__________________________________________________________________________________________________
normal_bn_1_15 (BatchNormalizat (None, 11, 11, 672) 2688 normal_conv_1_15[0][0]
__________________________________________________________________________________________________
activation_213 (Activation) (None, 11, 11, 672) 0 normal_bn_1_15[0][0]
__________________________________________________________________________________________________
activation_215 (Activation) (None, 11, 11, 672) 0 adjust_bn_15[0][0]
__________________________________________________________________________________________________
activation_217 (Activation) (None, 11, 11, 672) 0 adjust_bn_15[0][0]
__________________________________________________________________________________________________
activation_219 (Activation) (None, 11, 11, 672) 0 adjust_bn_15[0][0]
__________________________________________________________________________________________________
activation_221 (Activation) (None, 11, 11, 672) 0 normal_bn_1_15[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left1_1 (None, 11, 11, 672) 468384 activation_213[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right1_ (None, 11, 11, 672) 457632 activation_215[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left2_1 (None, 11, 11, 672) 468384 activation_217[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right2_ (None, 11, 11, 672) 457632 activation_219[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left5_1 (None, 11, 11, 672) 457632 activation_221[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_1_normal_left1_15[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_1_normal_right1_15
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_1_normal_left2_15[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_1_normal_right2_15
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_1_normal_left5_15[
__________________________________________________________________________________________________
activation_214 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_left1_
__________________________________________________________________________________________________
activation_216 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_right1
__________________________________________________________________________________________________
activation_218 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_left2_
__________________________________________________________________________________________________
activation_220 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_right2
__________________________________________________________________________________________________
activation_222 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_left5_
__________________________________________________________________________________________________
separable_conv_2_normal_left1_1 (None, 11, 11, 672) 468384 activation_214[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right1_ (None, 11, 11, 672) 457632 activation_216[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left2_1 (None, 11, 11, 672) 468384 activation_218[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right2_ (None, 11, 11, 672) 457632 activation_220[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left5_1 (None, 11, 11, 672) 457632 activation_222[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_2_normal_left1_15[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_2_normal_right1_15
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_2_normal_left2_15[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_2_normal_right2_15
__________________________________________________________________________________________________
normal_left3_15 (AveragePooling (None, 11, 11, 672) 0 normal_bn_1_15[0][0]
__________________________________________________________________________________________________
normal_left4_15 (AveragePooling (None, 11, 11, 672) 0 adjust_bn_15[0][0]
__________________________________________________________________________________________________
normal_right4_15 (AveragePoolin (None, 11, 11, 672) 0 adjust_bn_15[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_2_normal_left5_15[
__________________________________________________________________________________________________
normal_add_1_15 (Add) (None, 11, 11, 672) 0 separable_conv_2_bn_normal_left1_
separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________
normal_add_2_15 (Add) (None, 11, 11, 672) 0 separable_conv_2_bn_normal_left2_
separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________
normal_add_3_15 (Add) (None, 11, 11, 672) 0 normal_left3_15[0][0]
adjust_bn_15[0][0]
__________________________________________________________________________________________________
normal_add_4_15 (Add) (None, 11, 11, 672) 0 normal_left4_15[0][0]
normal_right4_15[0][0]
__________________________________________________________________________________________________
normal_add_5_15 (Add) (None, 11, 11, 672) 0 separable_conv_2_bn_normal_left5_
normal_bn_1_15[0][0]
__________________________________________________________________________________________________
normal_concat_15 (Concatenate) (None, 11, 11, 4032) 0 adjust_bn_15[0][0]
normal_add_1_15[0][0]
normal_add_2_15[0][0]
normal_add_3_15[0][0]
normal_add_4_15[0][0]
normal_add_5_15[0][0]
__________________________________________________________________________________________________
activation_223 (Activation) (None, 11, 11, 4032) 0 normal_concat_14[0][0]
__________________________________________________________________________________________________
activation_224 (Activation) (None, 11, 11, 4032) 0 normal_concat_15[0][0]
__________________________________________________________________________________________________
adjust_conv_projection_16 (Conv (None, 11, 11, 672) 2709504 activation_223[0][0]
__________________________________________________________________________________________________
normal_conv_1_16 (Conv2D) (None, 11, 11, 672) 2709504 activation_224[0][0]
__________________________________________________________________________________________________
adjust_bn_16 (BatchNormalizatio (None, 11, 11, 672) 2688 adjust_conv_projection_16[0][0]
__________________________________________________________________________________________________
normal_bn_1_16 (BatchNormalizat (None, 11, 11, 672) 2688 normal_conv_1_16[0][0]
__________________________________________________________________________________________________
activation_225 (Activation) (None, 11, 11, 672) 0 normal_bn_1_16[0][0]
__________________________________________________________________________________________________
activation_227 (Activation) (None, 11, 11, 672) 0 adjust_bn_16[0][0]
__________________________________________________________________________________________________
activation_229 (Activation) (None, 11, 11, 672) 0 adjust_bn_16[0][0]
__________________________________________________________________________________________________
activation_231 (Activation) (None, 11, 11, 672) 0 adjust_bn_16[0][0]
__________________________________________________________________________________________________
activation_233 (Activation) (None, 11, 11, 672) 0 normal_bn_1_16[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left1_1 (None, 11, 11, 672) 468384 activation_225[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right1_ (None, 11, 11, 672) 457632 activation_227[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left2_1 (None, 11, 11, 672) 468384 activation_229[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right2_ (None, 11, 11, 672) 457632 activation_231[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left5_1 (None, 11, 11, 672) 457632 activation_233[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_1_normal_left1_16[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_1_normal_right1_16
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_1_normal_left2_16[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_1_normal_right2_16
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_1_normal_left5_16[
__________________________________________________________________________________________________
activation_226 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_left1_
__________________________________________________________________________________________________
activation_228 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_right1
__________________________________________________________________________________________________
activation_230 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_left2_
__________________________________________________________________________________________________
activation_232 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_right2
__________________________________________________________________________________________________
activation_234 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_left5_
__________________________________________________________________________________________________
separable_conv_2_normal_left1_1 (None, 11, 11, 672) 468384 activation_226[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right1_ (None, 11, 11, 672) 457632 activation_228[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left2_1 (None, 11, 11, 672) 468384 activation_230[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right2_ (None, 11, 11, 672) 457632 activation_232[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left5_1 (None, 11, 11, 672) 457632 activation_234[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_2_normal_left1_16[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_2_normal_right1_16
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_2_normal_left2_16[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_2_normal_right2_16
__________________________________________________________________________________________________
normal_left3_16 (AveragePooling (None, 11, 11, 672) 0 normal_bn_1_16[0][0]
__________________________________________________________________________________________________
normal_left4_16 (AveragePooling (None, 11, 11, 672) 0 adjust_bn_16[0][0]
__________________________________________________________________________________________________
normal_right4_16 (AveragePoolin (None, 11, 11, 672) 0 adjust_bn_16[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_2_normal_left5_16[
__________________________________________________________________________________________________
normal_add_1_16 (Add) (None, 11, 11, 672) 0 separable_conv_2_bn_normal_left1_
separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________
normal_add_2_16 (Add) (None, 11, 11, 672) 0 separable_conv_2_bn_normal_left2_
separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________
normal_add_3_16 (Add) (None, 11, 11, 672) 0 normal_left3_16[0][0]
adjust_bn_16[0][0]
__________________________________________________________________________________________________
normal_add_4_16 (Add) (None, 11, 11, 672) 0 normal_left4_16[0][0]
normal_right4_16[0][0]
__________________________________________________________________________________________________
normal_add_5_16 (Add) (None, 11, 11, 672) 0 separable_conv_2_bn_normal_left5_
normal_bn_1_16[0][0]
__________________________________________________________________________________________________
normal_concat_16 (Concatenate) (None, 11, 11, 4032) 0 adjust_bn_16[0][0]
normal_add_1_16[0][0]
normal_add_2_16[0][0]
normal_add_3_16[0][0]
normal_add_4_16[0][0]
normal_add_5_16[0][0]
__________________________________________________________________________________________________
activation_235 (Activation) (None, 11, 11, 4032) 0 normal_concat_15[0][0]
__________________________________________________________________________________________________
activation_236 (Activation) (None, 11, 11, 4032) 0 normal_concat_16[0][0]
__________________________________________________________________________________________________
adjust_conv_projection_17 (Conv (None, 11, 11, 672) 2709504 activation_235[0][0]
__________________________________________________________________________________________________
normal_conv_1_17 (Conv2D) (None, 11, 11, 672) 2709504 activation_236[0][0]
__________________________________________________________________________________________________
adjust_bn_17 (BatchNormalizatio (None, 11, 11, 672) 2688 adjust_conv_projection_17[0][0]
__________________________________________________________________________________________________
normal_bn_1_17 (BatchNormalizat (None, 11, 11, 672) 2688 normal_conv_1_17[0][0]
__________________________________________________________________________________________________
activation_237 (Activation) (None, 11, 11, 672) 0 normal_bn_1_17[0][0]
__________________________________________________________________________________________________
activation_239 (Activation) (None, 11, 11, 672) 0 adjust_bn_17[0][0]
__________________________________________________________________________________________________
activation_241 (Activation) (None, 11, 11, 672) 0 adjust_bn_17[0][0]
__________________________________________________________________________________________________
activation_243 (Activation) (None, 11, 11, 672) 0 adjust_bn_17[0][0]
__________________________________________________________________________________________________
activation_245 (Activation) (None, 11, 11, 672) 0 normal_bn_1_17[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left1_1 (None, 11, 11, 672) 468384 activation_237[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right1_ (None, 11, 11, 672) 457632 activation_239[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left2_1 (None, 11, 11, 672) 468384 activation_241[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right2_ (None, 11, 11, 672) 457632 activation_243[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left5_1 (None, 11, 11, 672) 457632 activation_245[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_1_normal_left1_17[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_1_normal_right1_17
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_1_normal_left2_17[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_1_normal_right2_17
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_1_normal_left5_17[
__________________________________________________________________________________________________
activation_238 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_left1_
__________________________________________________________________________________________________
activation_240 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_right1
__________________________________________________________________________________________________
activation_242 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_left2_
__________________________________________________________________________________________________
activation_244 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_right2
__________________________________________________________________________________________________
activation_246 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_left5_
__________________________________________________________________________________________________
separable_conv_2_normal_left1_1 (None, 11, 11, 672) 468384 activation_238[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right1_ (None, 11, 11, 672) 457632 activation_240[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left2_1 (None, 11, 11, 672) 468384 activation_242[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right2_ (None, 11, 11, 672) 457632 activation_244[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left5_1 (None, 11, 11, 672) 457632 activation_246[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_2_normal_left1_17[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_2_normal_right1_17
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_2_normal_left2_17[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_2_normal_right2_17
__________________________________________________________________________________________________
normal_left3_17 (AveragePooling (None, 11, 11, 672) 0 normal_bn_1_17[0][0]
__________________________________________________________________________________________________
normal_left4_17 (AveragePooling (None, 11, 11, 672) 0 adjust_bn_17[0][0]
__________________________________________________________________________________________________
normal_right4_17 (AveragePoolin (None, 11, 11, 672) 0 adjust_bn_17[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_2_normal_left5_17[
__________________________________________________________________________________________________
normal_add_1_17 (Add) (None, 11, 11, 672) 0 separable_conv_2_bn_normal_left1_
separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________
normal_add_2_17 (Add) (None, 11, 11, 672) 0 separable_conv_2_bn_normal_left2_
separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________
normal_add_3_17 (Add) (None, 11, 11, 672) 0 normal_left3_17[0][0]
adjust_bn_17[0][0]
__________________________________________________________________________________________________
normal_add_4_17 (Add) (None, 11, 11, 672) 0 normal_left4_17[0][0]
normal_right4_17[0][0]
__________________________________________________________________________________________________
normal_add_5_17 (Add) (None, 11, 11, 672) 0 separable_conv_2_bn_normal_left5_
normal_bn_1_17[0][0]
__________________________________________________________________________________________________
normal_concat_17 (Concatenate) (None, 11, 11, 4032) 0 adjust_bn_17[0][0]
normal_add_1_17[0][0]
normal_add_2_17[0][0]
normal_add_3_17[0][0]
normal_add_4_17[0][0]
normal_add_5_17[0][0]
__________________________________________________________________________________________________
activation_247 (Activation) (None, 11, 11, 4032) 0 normal_concat_16[0][0]
__________________________________________________________________________________________________
activation_248 (Activation) (None, 11, 11, 4032) 0 normal_concat_17[0][0]
__________________________________________________________________________________________________
adjust_conv_projection_18 (Conv (None, 11, 11, 672) 2709504 activation_247[0][0]
__________________________________________________________________________________________________
normal_conv_1_18 (Conv2D) (None, 11, 11, 672) 2709504 activation_248[0][0]
__________________________________________________________________________________________________
adjust_bn_18 (BatchNormalizatio (None, 11, 11, 672) 2688 adjust_conv_projection_18[0][0]
__________________________________________________________________________________________________
normal_bn_1_18 (BatchNormalizat (None, 11, 11, 672) 2688 normal_conv_1_18[0][0]
__________________________________________________________________________________________________
activation_249 (Activation) (None, 11, 11, 672) 0 normal_bn_1_18[0][0]
__________________________________________________________________________________________________
activation_251 (Activation) (None, 11, 11, 672) 0 adjust_bn_18[0][0]
__________________________________________________________________________________________________
activation_253 (Activation) (None, 11, 11, 672) 0 adjust_bn_18[0][0]
__________________________________________________________________________________________________
activation_255 (Activation) (None, 11, 11, 672) 0 adjust_bn_18[0][0]
__________________________________________________________________________________________________
activation_257 (Activation) (None, 11, 11, 672) 0 normal_bn_1_18[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left1_1 (None, 11, 11, 672) 468384 activation_249[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right1_ (None, 11, 11, 672) 457632 activation_251[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left2_1 (None, 11, 11, 672) 468384 activation_253[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_right2_ (None, 11, 11, 672) 457632 activation_255[0][0]
__________________________________________________________________________________________________
separable_conv_1_normal_left5_1 (None, 11, 11, 672) 457632 activation_257[0][0]
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_1_normal_left1_18[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_1_normal_right1_18
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_1_normal_left2_18[
__________________________________________________________________________________________________
separable_conv_1_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_1_normal_right2_18
__________________________________________________________________________________________________
separable_conv_1_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_1_normal_left5_18[
__________________________________________________________________________________________________
activation_250 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_left1_
__________________________________________________________________________________________________
activation_252 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_right1
__________________________________________________________________________________________________
activation_254 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_left2_
__________________________________________________________________________________________________
activation_256 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_right2
__________________________________________________________________________________________________
activation_258 (Activation) (None, 11, 11, 672) 0 separable_conv_1_bn_normal_left5_
__________________________________________________________________________________________________
separable_conv_2_normal_left1_1 (None, 11, 11, 672) 468384 activation_250[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right1_ (None, 11, 11, 672) 457632 activation_252[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left2_1 (None, 11, 11, 672) 468384 activation_254[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_right2_ (None, 11, 11, 672) 457632 activation_256[0][0]
__________________________________________________________________________________________________
separable_conv_2_normal_left5_1 (None, 11, 11, 672) 457632 activation_258[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_2_normal_left1_18[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_2_normal_right1_18
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_2_normal_left2_18[
__________________________________________________________________________________________________
separable_conv_2_bn_normal_righ (None, 11, 11, 672) 2688 separable_conv_2_normal_right2_18
__________________________________________________________________________________________________
normal_left3_18 (AveragePooling (None, 11, 11, 672) 0 normal_bn_1_18[0][0]
__________________________________________________________________________________________________
normal_left4_18 (AveragePooling (None, 11, 11, 672) 0 adjust_bn_18[0][0]
__________________________________________________________________________________________________
normal_right4_18 (AveragePoolin (None, 11, 11, 672) 0 adjust_bn_18[0][0]
__________________________________________________________________________________________________
separable_conv_2_bn_normal_left (None, 11, 11, 672) 2688 separable_conv_2_normal_left5_18[
__________________________________________________________________________________________________
normal_add_1_18 (Add) (None, 11, 11, 672) 0 separable_conv_2_bn_normal_left1_
separable_conv_2_bn_normal_right1
__________________________________________________________________________________________________
normal_add_2_18 (Add) (None, 11, 11, 672) 0 separable_conv_2_bn_normal_left2_
separable_conv_2_bn_normal_right2
__________________________________________________________________________________________________
normal_add_3_18 (Add) (None, 11, 11, 672) 0 normal_left3_18[0][0]
adjust_bn_18[0][0]
__________________________________________________________________________________________________
normal_add_4_18 (Add) (None, 11, 11, 672) 0 normal_left4_18[0][0]
normal_right4_18[0][0]
__________________________________________________________________________________________________
normal_add_5_18 (Add) (None, 11, 11, 672) 0 separable_conv_2_bn_normal_left5_
normal_bn_1_18[0][0]
__________________________________________________________________________________________________
normal_concat_18 (Concatenate) (None, 11, 11, 4032) 0 adjust_bn_18[0][0]
normal_add_1_18[0][0]
normal_add_2_18[0][0]
normal_add_3_18[0][0]
normal_add_4_18[0][0]
normal_add_5_18[0][0]
__________________________________________________________________________________________________
activation_259 (Activation) (None, 11, 11, 4032) 0 normal_concat_18[0][0]
__________________________________________________________________________________________________
global_average_pooling2d (Globa (None, 4032) 0 activation_259[0][0]
==================================================================================================
Total params: 84,916,818
Trainable params: 84,720,150
Non-trainable params: 196,668
__________________________________________________________________________________________________
Ce modèle est bien plus complexe que les précédents et possède presque 85 millions de paramètres.
datagen = ImageDataGenerator(rescale=1/255,
validation_split=0.3)
train_data = datagen.flow_from_directory(batch_size=128,
directory=r'data',
shuffle=True,
seed = 1,
target_size=(331, 331),
color_mode="rgb",
subset="training",
class_mode='categorical'
)
validation_data = datagen.flow_from_directory(batch_size=128,
directory=r'data',
shuffle=False,
seed = 1,
target_size=(331, 331),
color_mode="rgb",
subset="validation",
class_mode='categorical')
Found 14458 images belonging to 120 classes. Found 6122 images belonging to 120 classes.
base_model.trainable = False
model = tf.keras.models.Sequential([
base_model,
Dense(512, activation='relu', kernel_initializer=he_uniform()),
Dropout(0.5),
Dense(512, activation='relu', kernel_initializer=he_uniform()),
Dropout(0.5),
Dense(len(races_list), activation='softmax')
])
model.compile(optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'])
log_dir = "logs/fit/TL_Model_NASNetLarge"
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
model.fit_generator(generator=train_data,
validation_data=validation_data,
steps_per_epoch=len(train_data),
validation_steps=len(validation_data),
epochs=10,
callbacks=[tensorboard_callback]
)
Epoch 1/10 113/113 [==============================] - 356s 3s/step - loss: 1.1248 - accuracy: 0.7679 - val_loss: 0.2611 - val_accuracy: 0.9327 Epoch 2/10 113/113 [==============================] - 351s 3s/step - loss: 0.3631 - accuracy: 0.9124 - val_loss: 0.2777 - val_accuracy: 0.9307 Epoch 3/10 113/113 [==============================] - 351s 3s/step - loss: 0.3022 - accuracy: 0.9228 - val_loss: 0.2774 - val_accuracy: 0.9291 Epoch 4/10 113/113 [==============================] - 350s 3s/step - loss: 0.2761 - accuracy: 0.9276 - val_loss: 0.2667 - val_accuracy: 0.9311 Epoch 5/10 113/113 [==============================] - 350s 3s/step - loss: 0.2545 - accuracy: 0.9312 - val_loss: 0.2745 - val_accuracy: 0.9337 Epoch 6/10 113/113 [==============================] - 350s 3s/step - loss: 0.2400 - accuracy: 0.9329 - val_loss: 0.2935 - val_accuracy: 0.9271 Epoch 7/10 113/113 [==============================] - 351s 3s/step - loss: 0.2255 - accuracy: 0.9395 - val_loss: 0.2993 - val_accuracy: 0.9285 Epoch 8/10 113/113 [==============================] - 351s 3s/step - loss: 0.2088 - accuracy: 0.9392 - val_loss: 0.2915 - val_accuracy: 0.9312 Epoch 9/10 113/113 [==============================] - 351s 3s/step - loss: 0.1914 - accuracy: 0.9436 - val_loss: 0.3144 - val_accuracy: 0.9262 Epoch 10/10 113/113 [==============================] - 350s 3s/step - loss: 0.1942 - accuracy: 0.9422 - val_loss: 0.3048 - val_accuracy: 0.9252
<tensorflow.python.keras.callbacks.History at 0x7f83f93089b0>
Dès le premier epoch, le modèle atteint 93% de réussite avec pour maximum 93.11% au quatrième epoch.
Testons avec de l'augmentation de données.
datagen = ImageDataGenerator(rescale=1/255,
validation_split=0.3,
zoom_range=0.1,
brightness_range=[0.7,1.2],
rotation_range=25,
horizontal_flip=True,
zca_whitening = True,
height_shift_range=0.05,
width_shift_range=0.05)
datagen_val = ImageDataGenerator(rescale=1/255,
validation_split=0.3)
train_data = datagen.flow_from_directory(batch_size=128,
directory=r'data',
shuffle=True,
seed = 1,
target_size=(224, 224),
color_mode="rgb",
subset="training",
class_mode='categorical'
)
validation_data = datagen_val.flow_from_directory(batch_size=128,
directory=r'data',
shuffle=True,
seed = 1,
target_size=(224, 224),
color_mode="rgb",
subset="validation",
class_mode='categorical')
base_model.trainable = False
model = tf.keras.models.Sequential([
base_model,
Dense(512, activation='relu', kernel_initializer=he_uniform()),
Dropout(0.5),
Dense(512, activation='relu', kernel_initializer=he_uniform()),
Dropout(0.5),
Dense(len(races_list), activation='softmax')
])
model.compile(optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'])
log_dir = "logs/fit/TL_Model_NASNetLarge_DA"
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
model.fit_generator(generator=train_data,
validation_data=validation_data,
steps_per_epoch=len(train_data),
validation_steps=len(validation_data),
epochs=100,
callbacks=[tf.keras.callbacks.EarlyStopping('val_loss', patience=3)]
)
Après 40 epochs, le meilleur résultat n'est pas meilleur que précédemment étant donné que nous obtenons maximum 91% de taux de succès
Nous allons maintenant effectuer une recherche d'hyperparmètres en jouant sur le nombre de neurones par couches, l'intensité de dropout, le coefficent d'apprentissage et le beta_1 utilisé pour l'optimizer Adam.
Nous emploierons un random search pour cela avec 10 itérations. Le temps d'éxecution étant assez lent, nous allons pas effectuer plus de recherches.
HP_NUM_UNITS_1 = hp.HParam('num_units_1', hp.Discrete([512, 1024, 2048]))
HP_NUM_UNITS_2 = hp.HParam('num_units_2', hp.Discrete([256, 512, 1024]))
HP_DROPOUT_1 = hp.HParam('dropout_1', hp.RealInterval(0.4, 0.6))
HP_DROPOUT_2 = hp.HParam('dropout_2', hp.RealInterval(0.4, 0.6))
HP_LR = hp.HParam('lr', hp.RealInterval(0.00001, 0.001))
HP_BETA = hp.HParam('beta', hp.RealInterval(0.9, 0.999))
METRIC_ACCURACY = 'accuracy'
with tf.summary.create_file_writer('logs/hparam_tuning').as_default():
hp.hparams_config(
hparams=[HP_NUM_UNITS_1, HP_NUM_UNITS_2, HP_DROPOUT_1, HP_DROPOUT_2, HP_LR, HP_BETA],
metrics=[hp.Metric(METRIC_ACCURACY, display_name='Accuracy')],
)
def random_model(hparams, i):
base_model.trainable = False
model = tf.keras.models.Sequential([
base_model,
Dense(hparams[HP_NUM_UNITS_1], activation='relu', kernel_initializer=he_uniform()),
Dropout(hparams[HP_DROPOUT_1]),
Dense(hparams[HP_NUM_UNITS_2], activation='relu', kernel_initializer=he_uniform()),
Dropout(hparams[HP_DROPOUT_2]),
Dense(len(races_list), activation='softmax')
])
model.compile(optimizer=Adam(learning_rate=hparams[HP_LR], beta_1=hparams[HP_BETA]),
loss='categorical_crossentropy',
metrics=['accuracy'])
history = model.fit_generator(generator=train_data,
validation_data=validation_data,
steps_per_epoch=len(train_data),
validation_steps=len(validation_data),
epochs=10,
callbacks=[tf.keras.callbacks.TensorBoard(log_dir="logs/fit/TL_Model_NASNetLarge_FT_"+str(i), histogram_freq=1),
hp.KerasCallback("logs/hparam_tuning/TL_Model_NASNetLarge_FT_"+str(i), hparams)])
return np.max(history.history['val_accuracy']), np.argmax(history.history['val_accuracy'])
best_dict = {'nb1': None, 'nb2': None, 'dp1': None, 'dp2': None, 'lr': None, 'beta': None}
min_acc = 0
ind_epoch = 0
it = None
for i in range(10):
print("Itération n°", i + 1)
nb_1 = np.random.choice(HP_NUM_UNITS_1.domain.values)
nb_2 = np.random.choice(HP_NUM_UNITS_2.domain.values)
drop_1 = np.random.uniform(HP_DROPOUT_1.domain.min_value, HP_DROPOUT_1.domain.max_value)
drop_2 = np.random.uniform(HP_DROPOUT_2.domain.min_value, HP_DROPOUT_2.domain.max_value)
lr = 10**np.random.uniform(np.log10(HP_LR.domain.max_value), np.log10(HP_LR.domain.min_value))
beta = 1-(10**np.random.uniform(np.log10(1-HP_BETA.domain.max_value), np.log10(1-HP_BETA.domain.min_value)))
print("Nb neurones 1 : ",nb_1)
print("Nb neurones 2 : ",nb_2)
print("Dropout 1 : ",drop_1)
print("Dropout 2 : ",drop_2)
print("Learning rate : ",lr)
print("Beta : ", beta)
hparams = {
HP_NUM_UNITS_1: nb_1,
HP_NUM_UNITS_2: nb_2,
HP_DROPOUT_1: drop_1,
HP_DROPOUT_2: drop_2,
HP_LR: lr,
HP_BETA: beta,
}
acc, ep = random_model(hparams, i)
if acc > min_acc:
best_dict['nb1'] = nb_1
best_dict['nb2'] = nb_2
best_dict['dp1'] = drop_1
best_dict['dp2'] = drop_2
best_dict['lr'] = lr
best_dict['beta'] = beta
ind_epoch = ep + 1
min_acc = acc
it = i + 1
print("Taux de succès : ",acc)
print("------------------")
print("Top accuracy : ", min_acc, " with ", ind_epoch, " epochs", " iteration : ", it)
print(best_dict)
Itération n° 1
Nb neurones 1 : 1024
Nb neurones 2 : 256
Dropout 1 : 0.5865114718677318
Dropout 2 : 0.42562488958587136
Learning rate : 1.0044283665868866e-05
Beta : 0.9970339535100061
WARNING:tensorflow:From <ipython-input-6-0b1b3bcd8be2>:21: Model.fit_generator (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.
Instructions for updating:
Please use Model.fit, which supports generators.
Epoch 1/10
113/113 [==============================] - 359s 3s/step - loss: 4.7563 - accuracy: 0.0184 - val_loss: 4.5304 - val_accuracy: 0.1790
Epoch 2/10
113/113 [==============================] - 350s 3s/step - loss: 4.5158 - accuracy: 0.0678 - val_loss: 4.2915 - val_accuracy: 0.5849
Epoch 3/10
113/113 [==============================] - 350s 3s/step - loss: 4.2856 - accuracy: 0.1648 - val_loss: 4.0260 - val_accuracy: 0.7886
Epoch 4/10
113/113 [==============================] - 350s 3s/step - loss: 4.0417 - accuracy: 0.2693 - val_loss: 3.7021 - val_accuracy: 0.8479
Epoch 5/10
113/113 [==============================] - 350s 3s/step - loss: 3.7621 - accuracy: 0.3701 - val_loss: 3.3176 - val_accuracy: 0.8737
Epoch 6/10
113/113 [==============================] - 350s 3s/step - loss: 3.4442 - accuracy: 0.4586 - val_loss: 2.8861 - val_accuracy: 0.8866
Epoch 7/10
113/113 [==============================] - 350s 3s/step - loss: 3.1131 - accuracy: 0.5133 - val_loss: 2.4322 - val_accuracy: 0.8930
Epoch 8/10
113/113 [==============================] - 349s 3s/step - loss: 2.7777 - accuracy: 0.5752 - val_loss: 1.9850 - val_accuracy: 0.8987
Epoch 9/10
113/113 [==============================] - 350s 3s/step - loss: 2.4196 - accuracy: 0.6333 - val_loss: 1.5734 - val_accuracy: 0.9067
Epoch 10/10
113/113 [==============================] - 350s 3s/step - loss: 2.1023 - accuracy: 0.6806 - val_loss: 1.2212 - val_accuracy: 0.9121
Taux de succès : 0.9121202230453491
Itération n° 2
Nb neurones 1 : 2048
Nb neurones 2 : 512
Dropout 1 : 0.4608657896491898
Dropout 2 : 0.44970655695131745
Learning rate : 2.0529182654224593e-05
Beta : 0.9908002890675373
Epoch 1/10
113/113 [==============================] - 359s 3s/step - loss: 4.3902 - accuracy: 0.1490 - val_loss: 3.7144 - val_accuracy: 0.8512
Epoch 2/10
113/113 [==============================] - 350s 3s/step - loss: 3.2412 - accuracy: 0.6143 - val_loss: 2.1387 - val_accuracy: 0.9064
Epoch 3/10
113/113 [==============================] - 350s 3s/step - loss: 1.8580 - accuracy: 0.8038 - val_loss: 0.7802 - val_accuracy: 0.9227
Epoch 4/10
113/113 [==============================] - 350s 3s/step - loss: 0.8992 - accuracy: 0.8768 - val_loss: 0.3337 - val_accuracy: 0.9303
Epoch 5/10
113/113 [==============================] - 351s 3s/step - loss: 0.5147 - accuracy: 0.9025 - val_loss: 0.2585 - val_accuracy: 0.9342
Epoch 6/10
113/113 [==============================] - 350s 3s/step - loss: 0.3887 - accuracy: 0.9144 - val_loss: 0.2455 - val_accuracy: 0.9338
Epoch 7/10
113/113 [==============================] - 350s 3s/step - loss: 0.3347 - accuracy: 0.9219 - val_loss: 0.2425 - val_accuracy: 0.9340
Epoch 8/10
113/113 [==============================] - 350s 3s/step - loss: 0.3057 - accuracy: 0.9232 - val_loss: 0.2400 - val_accuracy: 0.9345
Epoch 9/10
113/113 [==============================] - 350s 3s/step - loss: 0.2884 - accuracy: 0.9280 - val_loss: 0.2386 - val_accuracy: 0.9347
Epoch 10/10
113/113 [==============================] - 350s 3s/step - loss: 0.2748 - accuracy: 0.9299 - val_loss: 0.2376 - val_accuracy: 0.9350
Taux de succès : 0.9349885582923889
Itération n° 3
Nb neurones 1 : 1024
Nb neurones 2 : 256
Dropout 1 : 0.5431026056383901
Dropout 2 : 0.5762426862213614
Learning rate : 3.233407513987984e-05
Beta : 0.9206282561213679
Epoch 1/10
113/113 [==============================] - 357s 3s/step - loss: 4.5540 - accuracy: 0.0652 - val_loss: 4.0162 - val_accuracy: 0.7310
Epoch 2/10
113/113 [==============================] - 349s 3s/step - loss: 3.7118 - accuracy: 0.3007 - val_loss: 2.6387 - val_accuracy: 0.8922
Epoch 3/10
113/113 [==============================] - 349s 3s/step - loss: 2.6257 - accuracy: 0.5472 - val_loss: 1.2953 - val_accuracy: 0.9141
Epoch 4/10
113/113 [==============================] - 350s 3s/step - loss: 1.7285 - accuracy: 0.7186 - val_loss: 0.6280 - val_accuracy: 0.9232
Epoch 5/10
113/113 [==============================] - 350s 3s/step - loss: 1.2028 - accuracy: 0.8042 - val_loss: 0.3924 - val_accuracy: 0.9289
Epoch 6/10
113/113 [==============================] - 350s 3s/step - loss: 0.9148 - accuracy: 0.8427 - val_loss: 0.3078 - val_accuracy: 0.9306
Epoch 7/10
113/113 [==============================] - 350s 3s/step - loss: 0.7385 - accuracy: 0.8685 - val_loss: 0.2743 - val_accuracy: 0.9307
Epoch 8/10
113/113 [==============================] - 350s 3s/step - loss: 0.6476 - accuracy: 0.8814 - val_loss: 0.2590 - val_accuracy: 0.9327
Epoch 9/10
113/113 [==============================] - 350s 3s/step - loss: 0.5748 - accuracy: 0.8866 - val_loss: 0.2505 - val_accuracy: 0.9334
Epoch 10/10
113/113 [==============================] - 350s 3s/step - loss: 0.5276 - accuracy: 0.8953 - val_loss: 0.2467 - val_accuracy: 0.9342
Taux de succès : 0.9341718554496765
Itération n° 4
Nb neurones 1 : 512
Nb neurones 2 : 1024
Dropout 1 : 0.4277780320221328
Dropout 2 : 0.5368127132114661
Learning rate : 0.00046653773990436114
Beta : 0.9729622785019131
Epoch 1/10
113/113 [==============================] - 357s 3s/step - loss: 1.4914 - accuracy: 0.7092 - val_loss: 0.2646 - val_accuracy: 0.9281
Epoch 2/10
113/113 [==============================] - 349s 3s/step - loss: 0.3065 - accuracy: 0.9192 - val_loss: 0.2555 - val_accuracy: 0.9329
Epoch 3/10
113/113 [==============================] - 350s 3s/step - loss: 0.2620 - accuracy: 0.9281 - val_loss: 0.2571 - val_accuracy: 0.9320
Epoch 4/10
113/113 [==============================] - 349s 3s/step - loss: 0.2314 - accuracy: 0.9342 - val_loss: 0.2655 - val_accuracy: 0.9320
Epoch 5/10
113/113 [==============================] - 350s 3s/step - loss: 0.2184 - accuracy: 0.9393 - val_loss: 0.2711 - val_accuracy: 0.9312
Epoch 6/10
113/113 [==============================] - 350s 3s/step - loss: 0.2013 - accuracy: 0.9409 - val_loss: 0.2734 - val_accuracy: 0.9281
Epoch 7/10
113/113 [==============================] - 350s 3s/step - loss: 0.1919 - accuracy: 0.9438 - val_loss: 0.2677 - val_accuracy: 0.9320
Epoch 8/10
113/113 [==============================] - 350s 3s/step - loss: 0.1800 - accuracy: 0.9458 - val_loss: 0.2721 - val_accuracy: 0.9319
Epoch 9/10
113/113 [==============================] - 350s 3s/step - loss: 0.1691 - accuracy: 0.9483 - val_loss: 0.2860 - val_accuracy: 0.9317
Epoch 10/10
113/113 [==============================] - 350s 3s/step - loss: 0.1525 - accuracy: 0.9518 - val_loss: 0.2795 - val_accuracy: 0.9324
Taux de succès : 0.9328650832176208
Itération n° 5
Nb neurones 1 : 512
Nb neurones 2 : 256
Dropout 1 : 0.5294389177186928
Dropout 2 : 0.5172359379439729
Learning rate : 0.00047841760454605706
Beta : 0.9935605819829503
Epoch 1/10
113/113 [==============================] - 357s 3s/step - loss: 2.2118 - accuracy: 0.5438 - val_loss: 0.2674 - val_accuracy: 0.9267
Epoch 2/10
113/113 [==============================] - 349s 3s/step - loss: 0.4823 - accuracy: 0.8797 - val_loss: 0.2730 - val_accuracy: 0.9301
Epoch 3/10
113/113 [==============================] - 349s 3s/step - loss: 0.3563 - accuracy: 0.9125 - val_loss: 0.2656 - val_accuracy: 0.9332
Epoch 4/10
113/113 [==============================] - 349s 3s/step - loss: 0.3100 - accuracy: 0.9214 - val_loss: 0.2723 - val_accuracy: 0.9347
Epoch 5/10
113/113 [==============================] - 350s 3s/step - loss: 0.2912 - accuracy: 0.9281 - val_loss: 0.2730 - val_accuracy: 0.9342
Epoch 6/10
113/113 [==============================] - 350s 3s/step - loss: 0.2695 - accuracy: 0.9308 - val_loss: 0.2703 - val_accuracy: 0.9329
Epoch 7/10
113/113 [==============================] - 350s 3s/step - loss: 0.2420 - accuracy: 0.9335 - val_loss: 0.2681 - val_accuracy: 0.9340
Epoch 8/10
113/113 [==============================] - 349s 3s/step - loss: 0.2241 - accuracy: 0.9375 - val_loss: 0.2673 - val_accuracy: 0.9340
Epoch 9/10
113/113 [==============================] - 350s 3s/step - loss: 0.2177 - accuracy: 0.9370 - val_loss: 0.2711 - val_accuracy: 0.9342
Epoch 10/10
113/113 [==============================] - 350s 3s/step - loss: 0.2062 - accuracy: 0.9393 - val_loss: 0.2678 - val_accuracy: 0.9343
Taux de succès : 0.934661865234375
Itération n° 6
Nb neurones 1 : 512
Nb neurones 2 : 256
Dropout 1 : 0.49767431845903976
Dropout 2 : 0.49720204652898226
Learning rate : 0.0003288175824895823
Beta : 0.9983812828958174
Epoch 1/10
113/113 [==============================] - 357s 3s/step - loss: 2.5164 - accuracy: 0.5069 - val_loss: 0.2883 - val_accuracy: 0.9260
Epoch 2/10
113/113 [==============================] - 350s 3s/step - loss: 0.5474 - accuracy: 0.8698 - val_loss: 0.2716 - val_accuracy: 0.9301
Epoch 3/10
113/113 [==============================] - 350s 3s/step - loss: 0.3833 - accuracy: 0.9064 - val_loss: 0.2702 - val_accuracy: 0.9325
Epoch 4/10
113/113 [==============================] - 350s 3s/step - loss: 0.3266 - accuracy: 0.9180 - val_loss: 0.2723 - val_accuracy: 0.9324
Epoch 5/10
113/113 [==============================] - 350s 3s/step - loss: 0.3128 - accuracy: 0.9206 - val_loss: 0.2743 - val_accuracy: 0.9320
Epoch 6/10
113/113 [==============================] - 350s 3s/step - loss: 0.2887 - accuracy: 0.9271 - val_loss: 0.2762 - val_accuracy: 0.9325
Epoch 7/10
113/113 [==============================] - 350s 3s/step - loss: 0.2733 - accuracy: 0.9279 - val_loss: 0.2795 - val_accuracy: 0.9342
Epoch 8/10
113/113 [==============================] - 350s 3s/step - loss: 0.2588 - accuracy: 0.9322 - val_loss: 0.2817 - val_accuracy: 0.9342
Epoch 9/10
113/113 [==============================] - 350s 3s/step - loss: 0.2445 - accuracy: 0.9344 - val_loss: 0.2805 - val_accuracy: 0.9338
Epoch 10/10
113/113 [==============================] - 350s 3s/step - loss: 0.2314 - accuracy: 0.9376 - val_loss: 0.2792 - val_accuracy: 0.9329
Taux de succès : 0.9341718554496765
Itération n° 7
Nb neurones 1 : 512
Nb neurones 2 : 1024
Dropout 1 : 0.4545278826265517
Dropout 2 : 0.5390753695265337
Learning rate : 9.646196753480824e-05
Beta : 0.9948854093868351
Epoch 1/10
113/113 [==============================] - 358s 3s/step - loss: 3.7665 - accuracy: 0.3321 - val_loss: 1.9097 - val_accuracy: 0.9054
Epoch 2/10
113/113 [==============================] - 349s 3s/step - loss: 1.3959 - accuracy: 0.7717 - val_loss: 0.2820 - val_accuracy: 0.9293
Epoch 3/10
113/113 [==============================] - 350s 3s/step - loss: 0.5145 - accuracy: 0.8790 - val_loss: 0.2487 - val_accuracy: 0.9338
Epoch 4/10
113/113 [==============================] - 349s 3s/step - loss: 0.3630 - accuracy: 0.9063 - val_loss: 0.2587 - val_accuracy: 0.9337
Epoch 5/10
113/113 [==============================] - 350s 3s/step - loss: 0.3098 - accuracy: 0.9166 - val_loss: 0.2514 - val_accuracy: 0.9332
Epoch 6/10
113/113 [==============================] - 350s 3s/step - loss: 0.2787 - accuracy: 0.9234 - val_loss: 0.2469 - val_accuracy: 0.9338
Epoch 7/10
113/113 [==============================] - 350s 3s/step - loss: 0.2684 - accuracy: 0.9283 - val_loss: 0.2448 - val_accuracy: 0.9330
Epoch 8/10
113/113 [==============================] - 350s 3s/step - loss: 0.2535 - accuracy: 0.9312 - val_loss: 0.2450 - val_accuracy: 0.9335
Epoch 9/10
113/113 [==============================] - 350s 3s/step - loss: 0.2379 - accuracy: 0.9337 - val_loss: 0.2466 - val_accuracy: 0.9345
Epoch 10/10
113/113 [==============================] - 350s 3s/step - loss: 0.2332 - accuracy: 0.9350 - val_loss: 0.2510 - val_accuracy: 0.9347
Taux de succès : 0.934661865234375
Itération n° 8
Nb neurones 1 : 1024
Nb neurones 2 : 1024
Dropout 1 : 0.5315642886427299
Dropout 2 : 0.48721037247520643
Learning rate : 1.4083532169160651e-05
Beta : 0.9470630395660633
Epoch 1/10
113/113 [==============================] - 357s 3s/step - loss: 4.6432 - accuracy: 0.0388 - val_loss: 4.2684 - val_accuracy: 0.4567
Epoch 2/10
113/113 [==============================] - 350s 3s/step - loss: 4.1419 - accuracy: 0.1946 - val_loss: 3.6541 - val_accuracy: 0.8514
Epoch 3/10
113/113 [==============================] - 350s 3s/step - loss: 3.5507 - accuracy: 0.4286 - val_loss: 2.8259 - val_accuracy: 0.9040
Epoch 4/10
113/113 [==============================] - 350s 3s/step - loss: 2.8296 - accuracy: 0.6221 - val_loss: 1.8621 - val_accuracy: 0.9151
Epoch 5/10
113/113 [==============================] - 350s 3s/step - loss: 2.0506 - accuracy: 0.7556 - val_loss: 1.0533 - val_accuracy: 0.9214
Epoch 6/10
113/113 [==============================] - 350s 3s/step - loss: 1.4338 - accuracy: 0.8305 - val_loss: 0.5925 - val_accuracy: 0.9273
Epoch 7/10
113/113 [==============================] - 350s 3s/step - loss: 1.0382 - accuracy: 0.8588 - val_loss: 0.3968 - val_accuracy: 0.9291
Epoch 8/10
113/113 [==============================] - 350s 3s/step - loss: 0.7897 - accuracy: 0.8804 - val_loss: 0.3149 - val_accuracy: 0.9298
Epoch 9/10
113/113 [==============================] - 350s 3s/step - loss: 0.6572 - accuracy: 0.8915 - val_loss: 0.2788 - val_accuracy: 0.9311
Epoch 10/10
113/113 [==============================] - 350s 3s/step - loss: 0.5615 - accuracy: 0.8967 - val_loss: 0.2601 - val_accuracy: 0.9307
Taux de succès : 0.9310683012008667
Itération n° 9
Nb neurones 1 : 2048
Nb neurones 2 : 256
Dropout 1 : 0.46875002212336847
Dropout 2 : 0.5094590917399153
Learning rate : 0.0004031307541486248
Beta : 0.9957314243700366
Epoch 1/10
113/113 [==============================] - 358s 3s/step - loss: 1.2731 - accuracy: 0.7484 - val_loss: 0.2819 - val_accuracy: 0.9273
Epoch 2/10
113/113 [==============================] - 350s 3s/step - loss: 0.3401 - accuracy: 0.9138 - val_loss: 0.2564 - val_accuracy: 0.9320
Epoch 3/10
113/113 [==============================] - 350s 3s/step - loss: 0.2963 - accuracy: 0.9236 - val_loss: 0.2681 - val_accuracy: 0.9329
Epoch 4/10
113/113 [==============================] - 350s 3s/step - loss: 0.2582 - accuracy: 0.9325 - val_loss: 0.2640 - val_accuracy: 0.9332
Epoch 5/10
113/113 [==============================] - 350s 3s/step - loss: 0.2308 - accuracy: 0.9365 - val_loss: 0.2646 - val_accuracy: 0.9337
Epoch 6/10
113/113 [==============================] - 350s 3s/step - loss: 0.2154 - accuracy: 0.9410 - val_loss: 0.2640 - val_accuracy: 0.9343
Epoch 7/10
113/113 [==============================] - 350s 3s/step - loss: 0.1952 - accuracy: 0.9438 - val_loss: 0.2678 - val_accuracy: 0.9330
Epoch 8/10
113/113 [==============================] - 350s 3s/step - loss: 0.1804 - accuracy: 0.9458 - val_loss: 0.2613 - val_accuracy: 0.9319
Epoch 9/10
113/113 [==============================] - 350s 3s/step - loss: 0.1659 - accuracy: 0.9508 - val_loss: 0.2694 - val_accuracy: 0.9324
Epoch 10/10
113/113 [==============================] - 350s 3s/step - loss: 0.1587 - accuracy: 0.9491 - val_loss: 0.2794 - val_accuracy: 0.9309
Taux de succès : 0.9343351721763611
Itération n° 10
Nb neurones 1 : 2048
Nb neurones 2 : 512
Dropout 1 : 0.41544219687963596
Dropout 2 : 0.5058681602404208
Learning rate : 1.3406581390425323e-05
Beta : 0.9960512858197407
Epoch 1/10
113/113 [==============================] - 358s 3s/step - loss: 4.5410 - accuracy: 0.0721 - val_loss: 4.0850 - val_accuracy: 0.7310
Epoch 2/10
113/113 [==============================] - 350s 3s/step - loss: 3.8519 - accuracy: 0.3933 - val_loss: 3.2433 - val_accuracy: 0.8971
Epoch 3/10
113/113 [==============================] - 350s 3s/step - loss: 3.0271 - accuracy: 0.6648 - val_loss: 2.1889 - val_accuracy: 0.9141
Epoch 4/10
113/113 [==============================] - 350s 3s/step - loss: 2.1249 - accuracy: 0.7747 - val_loss: 1.2133 - val_accuracy: 0.9193
Epoch 5/10
113/113 [==============================] - 350s 3s/step - loss: 1.3540 - accuracy: 0.8391 - val_loss: 0.6092 - val_accuracy: 0.9255
Epoch 6/10
113/113 [==============================] - 350s 3s/step - loss: 0.8770 - accuracy: 0.8645 - val_loss: 0.3588 - val_accuracy: 0.9291
Epoch 7/10
113/113 [==============================] - 350s 3s/step - loss: 0.5953 - accuracy: 0.8905 - val_loss: 0.2766 - val_accuracy: 0.9304
Epoch 8/10
113/113 [==============================] - 350s 3s/step - loss: 0.4590 - accuracy: 0.9032 - val_loss: 0.2510 - val_accuracy: 0.9317
Epoch 9/10
113/113 [==============================] - 350s 3s/step - loss: 0.3863 - accuracy: 0.9083 - val_loss: 0.2428 - val_accuracy: 0.9343
Epoch 10/10
113/113 [==============================] - 350s 3s/step - loss: 0.3366 - accuracy: 0.9193 - val_loss: 0.2401 - val_accuracy: 0.9337
Taux de succès : 0.9343351721763611
------------------
Top accuracy : 0.9349885582923889 with 10 epochs iteration : 2
{'nb1': 2048, 'nb2': 512, 'dp1': 0.4608657896491898, 'dp2': 0.44970655695131745, 'lr': 2.0529182654224593e-05, 'beta': 0.9908002890675373}
Nous pouvons atteindre 93.5% de succès en jouant avec le hyperparamètre.
En observant de plus près les résultats du modèle fournissant ce taux de réussite, nous pouvons noter que le modèle peut potentiellement avoir un meilleur succès. Ainsi, nous allons le reprendre et le ré-entraîner avec les mêmes hyperparamètres.
base_model.trainable = False
model = tf.keras.models.Sequential([
base_model,
Dense(2048, activation='relu', kernel_initializer=he_uniform()),
Dropout(0.46),
Dense(512, activation='relu', kernel_initializer=he_uniform()),
Dropout(0.45),
Dense(len(races_list), activation='softmax')
])
model.compile(optimizer=Adam(learning_rate=0.00002053, beta_1=0.9908),
loss='categorical_crossentropy',
metrics=['accuracy'])
log_dir = "logs/fit/TL_Model_NASNetLarge_FT"
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
model.fit_generator(generator=train_data,
validation_data=validation_data,
steps_per_epoch=len(train_data),
validation_steps=len(validation_data),
epochs=20,
callbacks=[tensorboard_callback]
)
Epoch 1/20 113/113 [==============================] - 358s 3s/step - loss: 4.3808 - accuracy: 0.1657 - val_loss: 3.6907 - val_accuracy: 0.8533 Epoch 2/20 113/113 [==============================] - 352s 3s/step - loss: 3.2160 - accuracy: 0.6160 - val_loss: 2.0791 - val_accuracy: 0.9007 Epoch 3/20 113/113 [==============================] - 352s 3s/step - loss: 1.7996 - accuracy: 0.8039 - val_loss: 0.7397 - val_accuracy: 0.9203 Epoch 4/20 113/113 [==============================] - 352s 3s/step - loss: 0.8522 - accuracy: 0.8797 - val_loss: 0.3260 - val_accuracy: 0.9286 Epoch 5/20 113/113 [==============================] - 352s 3s/step - loss: 0.4989 - accuracy: 0.9039 - val_loss: 0.2578 - val_accuracy: 0.9332 Epoch 6/20 113/113 [==============================] - 352s 3s/step - loss: 0.3798 - accuracy: 0.9155 - val_loss: 0.2440 - val_accuracy: 0.9350 Epoch 7/20 113/113 [==============================] - 352s 3s/step - loss: 0.3392 - accuracy: 0.9188 - val_loss: 0.2414 - val_accuracy: 0.9343 Epoch 8/20 113/113 [==============================] - 352s 3s/step - loss: 0.3022 - accuracy: 0.9261 - val_loss: 0.2403 - val_accuracy: 0.9353 Epoch 9/20 113/113 [==============================] - 352s 3s/step - loss: 0.2898 - accuracy: 0.9273 - val_loss: 0.2388 - val_accuracy: 0.9353 Epoch 10/20 113/113 [==============================] - 351s 3s/step - loss: 0.2760 - accuracy: 0.9277 - val_loss: 0.2372 - val_accuracy: 0.9348 Epoch 11/20 113/113 [==============================] - 351s 3s/step - loss: 0.2788 - accuracy: 0.9271 - val_loss: 0.2354 - val_accuracy: 0.9355 Epoch 12/20 113/113 [==============================] - 351s 3s/step - loss: 0.2489 - accuracy: 0.9335 - val_loss: 0.2358 - val_accuracy: 0.9352 Epoch 13/20 113/113 [==============================] - 351s 3s/step - loss: 0.2541 - accuracy: 0.9343 - val_loss: 0.2367 - val_accuracy: 0.9361 Epoch 14/20 113/113 [==============================] - 351s 3s/step - loss: 0.2405 - accuracy: 0.9381 - val_loss: 0.2368 - val_accuracy: 0.9353 Epoch 15/20 113/113 [==============================] - 351s 3s/step - loss: 0.2383 - accuracy: 0.9357 - val_loss: 0.2357 - val_accuracy: 0.9345 Epoch 16/20 113/113 [==============================] - 351s 3s/step - loss: 0.2251 - accuracy: 0.9374 - val_loss: 0.2362 - val_accuracy: 0.9352 Epoch 17/20 113/113 [==============================] - 351s 3s/step - loss: 0.2313 - accuracy: 0.9355 - val_loss: 0.2368 - val_accuracy: 0.9361 Epoch 18/20 113/113 [==============================] - 351s 3s/step - loss: 0.2119 - accuracy: 0.9403 - val_loss: 0.2361 - val_accuracy: 0.9347 Epoch 19/20 113/113 [==============================] - 351s 3s/step - loss: 0.2114 - accuracy: 0.9407 - val_loss: 0.2364 - val_accuracy: 0.9348 Epoch 20/20 113/113 [==============================] - 351s 3s/step - loss: 0.2110 - accuracy: 0.9397 - val_loss: 0.2364 - val_accuracy: 0.9353
<tensorflow.python.keras.callbacks.History at 0x7f9008e69b70>
Nous obtenons au mieux 93.61%.
Sauvegardons le modèle.
model.save('TL_Model_NASNetLarge_FT.h5')
En fonction du dernier modèle, nous allons regarder les différentes erreurs qu'il a effectué.
datagen = ImageDataGenerator(rescale=1/255,
validation_split=0.3)
validation_data = datagen.flow_from_directory(batch_size=1,
directory=r'data',
shuffle=False,
target_size=(331, 331),
color_mode="rgb",
subset="validation",
class_mode='categorical')
predictions = model.predict_generator(validation_data, steps=len(validation_data.classes))
predictions
Found 6122 images belonging to 120 classes.
array([[9.9942708e-01, 9.4085244e-06, 1.6468196e-06, ..., 2.4645421e-06,
2.2143836e-06, 1.3158597e-06],
[9.9963975e-01, 5.8137025e-06, 5.3983007e-07, ..., 1.5148343e-06,
1.2255189e-06, 1.0724281e-06],
[9.9975687e-01, 3.5442288e-06, 4.3988433e-07, ..., 9.4073613e-07,
9.3047964e-07, 7.5087911e-07],
...,
[1.9993195e-05, 3.1564518e-06, 5.6991554e-05, ..., 1.2809741e-06,
1.8285651e-05, 9.9530786e-01],
[3.4194447e-05, 4.8655365e-06, 4.4934059e-04, ..., 1.5929359e-06,
2.3835513e-05, 9.9501711e-01],
[5.2390940e-05, 1.0177098e-05, 3.0687530e-04, ..., 5.2179189e-06,
3.1307813e-05, 9.8480475e-01]], dtype=float32)
Vérifions que le taux de succès soit bien cohérent
list_race = np.array([race for race in sorted(os.listdir("data"))])
y = validation_data.classes
predictions_sup = np.argmax(predictions, axis=1)
incorrect_pred = np.where(y != predictions_sup)[0]
np.mean(y == predictions_sup)
0.9353152564521399
Trions dans l'ordre décroissant les races de chiens ayant eu le plus d'erreur de prédiction
unique, counts = np.unique(y[incorrect_pred], return_counts=True)
incorrect_dict = dict(zip(unique, counts))
[(it[0],list_race[it[0]], it[1]) for it in sorted(incorrect_dict.items(), key=lambda x: x[1], reverse=True)]
[(24, 'Eskimo_dog', 19), (89, 'collie', 19), (20, 'English_foxhound', 16), (63, 'Shih', 12), (64, 'Siberian_husky', 12), (114, 'standard_schnauzer', 11), (115, 'toy_poodle', 11), (104, 'miniature_poodle', 10), (3, 'American_Staffordshire_terrier', 9), (4, 'Appenzeller', 9), (43, 'Lhasa', 9), (47, 'Norfolk_terrier', 8), (31, 'Greater_Swiss_Mountain_dog', 7), (68, 'Tibetan_terrier', 7), (119, 'wire', 7), (5, 'Australian_terrier', 6), (23, 'EntleBucher', 6), (41, 'Lakeland_terrier', 6), (62, 'Shetland_sheepdog', 6), (65, 'Staffordshire_bullterrier', 6), (73, 'Yorkshire_terrier', 6), (92, 'dingo', 6), (111, 'silky_terrier', 6), (118, 'whippet', 6), (36, 'Irish_wolfhound', 5), (46, 'Newfoundland', 5), (49, 'Norwich_terrier', 5), (54, 'Rhodesian_ridgeback', 5), (69, 'Walker_hound', 5), (101, 'malamute', 5), (116, 'toy_terrier', 5), (15, 'Cardigan', 4), (34, 'Irish_terrier', 4), (40, 'Labrador_retriever', 4), (44, 'Maltese_dog', 4), (59, 'Scotch_terrier', 4), (66, 'Sussex_spaniel', 4), (94, 'giant_schnauzer', 4), (98, 'kelpie', 4), (103, 'miniature_pinscher', 4), (105, 'miniature_schnauzer', 4), (106, 'otterhound', 4), (109, 'redbone', 4), (21, 'English_setter', 3), (29, 'Great_Dane', 3), (30, 'Great_Pyrenees', 3), (60, 'Scottish_deerhound', 3), (82, 'boxer', 3), (83, 'briard', 3), (117, 'vizsla', 3), (0, 'Afghan_hound', 2), (12, 'Bouvier_des_Flandres', 2), (13, 'Brabancon_griffon', 2), (14, 'Brittany_spaniel', 2), (16, 'Chesapeake_Bay_retriever', 2), (17, 'Chihuahua', 2), (22, 'English_springer', 2), (38, 'Japanese_spaniel', 2), (39, 'Kerry_blue_terrier', 2), (51, 'Pekinese', 2), (57, 'Saluki', 2), (58, 'Samoyed', 2), (61, 'Sealyham_terrier', 2), (67, 'Tibetan_mastiff', 2), (70, 'Weimaraner', 2), (72, 'West_Highland_white_terrier', 2), (75, 'basenji', 2), (78, 'black', 2), (85, 'cairn', 2), (91, 'dhole', 2), (100, 'kuvasz', 2), (113, 'standard_poodle', 2), (2, 'Airedale', 1), (7, 'Bernese_mountain_dog', 1), (8, 'Blenheim_spaniel', 1), (9, 'Border_collie', 1), (11, 'Boston_bull', 1), (25, 'French_bulldog', 1), (32, 'Ibizan_hound', 1), (37, 'Italian_greyhound', 1), (48, 'Norwegian_elkhound', 1), (52, 'Pembroke', 1), (53, 'Pomeranian', 1), (55, 'Rottweiler', 1), (71, 'Welsh_springer_spaniel', 1), (76, 'basset', 1), (80, 'bluetick', 1), (81, 'borzoi', 1), (84, 'bull_mastiff', 1), (87, 'clumber', 1), (88, 'cocker_spaniel', 1), (90, 'curly', 1), (93, 'flat', 1), (95, 'golden_retriever', 1), (96, 'groenendael', 1), (97, 'keeshond', 1), (102, 'malinois', 1), (107, 'papillon', 1), (108, 'pug', 1), (112, 'soft', 1)]
Regardons les mauvaises prédictions de la race Eskimo_dog
plt.figure(figsize=(15,30))
j = 0
for i in incorrect_pred[np.where(y[incorrect_pred] == 24)[0]]:
plt.subplot(10, 4, j+1)
plt.imshow(mpimg.imread('data/'+validation_data.filenames[i]))
plt.title(list_race[predictions_sup[i]])
plt.axis('off')
j += 1
plt.imshow(mpimg.imread('data/Siberian_husky/n02110185_10047.jpg'))
<matplotlib.image.AxesImage at 0x7f8f60e94908>
Nous voyons que la plupart des mauvaises prédictions font référence aux Husky sibériens qui est une race de chien leur ressemblant fortement.
Parmi les mauvaises prédictions, un dingo a été annoté sûrement dû à la couleur de son pelage.
Regardons maintenant pour la race 'collie'
plt.figure(figsize=(15,30))
j = 0
for i in incorrect_pred[np.where(y[incorrect_pred] == 89)[0]]:
plt.subplot(10, 4, j+1)
plt.imshow(mpimg.imread('data/'+validation_data.filenames[i]))
plt.title(list_race[predictions_sup[i]])
plt.axis('off')
j += 1
plt.imshow(mpimg.imread('data/Border_collie/n02106166_1032.jpg'))
<matplotlib.image.AxesImage at 0x7f8f6083c9b0>
Les mauvaises prédictions pour cette race semble être juste étant donné qu'elles sont pour la plupart des Border collie.
En explorant les données, beaucoup d'images pour la race collie ont été mal annotées.